-- Logs begin at Thu 2022-07-21 04:59:27 UTC, end at Thu 2022-07-21 06:15:37 UTC. -- Jul 21 05:29:27.767135 dv-cinder-850449-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 21 05:29:27.777670 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 21 05:29:27.779806 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 21 05:29:27 2022] *** Jul 21 05:29:27.779806 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 21 05:29:27.779806 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 21 05:29:27.779806 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: nodename: dv-cinder-850449-1-iscsi Jul 21 05:29:27.779806 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: machine: x86_64 Jul 21 05:29:27.779806 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: clock source: unix Jul 21 05:29:27.779806 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: pcre jit disabled Jul 21 05:29:27.779806 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: detected number of CPU cores: 4 Jul 21 05:29:27.779806 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: current working directory: / Jul 21 05:29:27.779806 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: detected binary path: /usr/bin/uwsgi-core Jul 21 05:29:27.779806 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: your processes number limit is 47680 Jul 21 05:29:27.779806 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: your memory page size is 4096 bytes Jul 21 05:29:27.779806 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: detected max file descriptor number: 2048 Jul 21 05:29:27.779806 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: lock engine: pthread robust mutexes Jul 21 05:29:27.780578 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: thunder lock: enabled Jul 21 05:29:27.780578 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 21 05:29:27.780578 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 21 05:29:27.826845 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: Python main interpreter initialized at 0x55c4aef169c0 Jul 21 05:29:27.826845 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: python threads support enabled Jul 21 05:29:27.826845 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: your server socket listen backlog is limited to 100 connections Jul 21 05:29:27.826845 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: your mercy for graceful operations on workers is 90 seconds Jul 21 05:29:27.827238 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: mapped 403077 bytes (393 KB) for 2 cores Jul 21 05:29:27.827303 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: *** Operational MODE: preforking *** Jul 21 05:29:27.827581 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: *** uWSGI is running in multiple interpreter mode *** Jul 21 05:29:27.827581 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: spawned uWSGI master process (pid: 80380) Jul 21 05:29:27.827790 dv-cinder-850449-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 21 05:29:27.828578 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: spawned uWSGI worker 1 (pid: 80390, cores: 1) Jul 21 05:29:27.828675 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: spawned uWSGI worker 2 (pid: 80391, cores: 1) Jul 21 05:29:27.830697 dv-cinder-850449-1-iscsi devstack@keystone.service[80380]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 21 05:29:33.143364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 05:29:33.143364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: from cryptography.utils import int_from_bytes Jul 21 05:29:33.144196 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 05:29:33.144196 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: from cryptography.utils import int_from_bytes Jul 21 05:29:33.146675 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 05:29:33.146675 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: from cryptography.utils import int_from_bytes Jul 21 05:29:33.147203 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 21 05:29:33.147203 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: from cryptography.utils import int_from_bytes Jul 21 05:29:33.192471 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 05:29:33.192646 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 05:29:33.192852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] command line args: [] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 05:29:33.193116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 05:29:33.193248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 21 05:29:33.193341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ================================================================================ {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 05:29:33.193410 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 21 05:29:33.193480 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] command line args: [] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 21 05:29:33.193539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] admin_token = **** {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.193741 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] config_dir = [] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.193840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 21 05:29:33.193928 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.194003 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ================================================================================ {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 21 05:29:33.194064 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] config_source = [] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.194183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] admin_token = **** {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.194271 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] debug = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.194443 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.195839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] config_dir = [] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.195839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.195963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.196033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] config_source = [] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.196096 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.196167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] debug = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.196243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.196306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.196389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.196462 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.196529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] list_limit = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.196589 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.196646 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] log_config_append = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.196703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.196766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.196766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] log_dir = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.196888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.196952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] log_file = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.197011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] list_limit = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.197068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.197125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] log_config_append = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.197190 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.197249 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.197249 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] log_dir = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.197351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.197409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] log_file = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.197471 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.197541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.197606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.197606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.197719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.197778 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.198064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.198155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.198281 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.198364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.198473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.198473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.198673 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.198795 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.198872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.198986 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.199129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.199251 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.199338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.199418 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.199535 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.199715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.199865 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.199949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.200243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.200417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.200417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] publish_errors = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.200621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.200751 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.200835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.200909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.201025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.201246 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.201368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.201468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.201550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] publish_errors = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.201699 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] standard_threads = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.201823 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.201930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.202014 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.202157 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.202282 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.202361 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.202495 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.202571 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] use_journal = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.202643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] use_json = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.202720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.202797 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] use_stderr = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.202971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] use_syslog = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.203136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] standard_threads = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.203227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.203308 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.203464 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.203590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.203741 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.203872 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.203989 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.204110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] use_journal = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.204224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.204338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] use_json = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.204338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] use_stderr = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.204563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.204683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] use_syslog = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.204797 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.204918 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 21 05:29:33.205031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.205031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.205253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.205370 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.205491 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.205607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.205721 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.205842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.205956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.206068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.206188 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.206301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.206414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.206539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.206655 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.206767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.206886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.207000 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.207147 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.207274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.207390 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.207515 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.207629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.207743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.207862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.208060 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.208184 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.208299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.208414 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.208527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.208640 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.208765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.208879 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.208992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.209105 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.209225 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.209344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.209471 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.209586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.209699 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.209813 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.209931 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.210045 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.210166 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.210287 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.210401 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.210584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.210753 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.210880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.211068 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.211192 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.211306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.211419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.211532 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.211645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.211774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.211891 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.212004 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.212116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.212237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.212359 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.212481 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.212595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.212751 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.212906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.213023 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.213147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.213269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.213383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.213495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.213614 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.213727 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.213924 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.214042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.214162 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.214276 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.214389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.214508 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.214630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.214630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.214846 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.214959 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.215096 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.215210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.215340 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.215464 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.215578 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.215691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.215803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.215922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.216045 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.216167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.216266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.216360 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.216455 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.216550 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.216655 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.216753 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.216849 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.216944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.217047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.217176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.217320 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.217437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.217550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.217661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.217783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.217904 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.218079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.218189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.218302 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.218421 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.218538 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.218651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.218773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.218897 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.219027 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.219156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.219275 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.219389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.219502 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.219623 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.219739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.219859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.219973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.220086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.220192 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.220314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.220436 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.220551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.220664 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.220777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.220890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.221016 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.221131 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.221245 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.221360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.221464 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.221566 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.221685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.221808 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.221930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.222044 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.222167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.222282 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.222405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.222520 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.222633 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.222746 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.222859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.222981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.223161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.223279 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.223393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.223507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.223628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.223743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.223865 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.223981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.224094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.224207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.224320 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.224433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.224560 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.224676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.224795 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.224910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.225023 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.225146 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.225335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.225453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.225568 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.225681 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.225794 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.225907 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.226029 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.226154 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.226285 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.226407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.226522 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.226636 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.226756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.226871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.226993 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.227123 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.227238 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.227351 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.227465 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.227580 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.227711 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] auth.external = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.227828 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.227941 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.228054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.228168 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.228296 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.228423 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.228539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.228652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] auth.password = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.228767 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.228880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] auth.token = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.229001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.229124 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.229246 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.229361 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.229475 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.229589 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.229710 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.229856 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.229981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.230106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.230223 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.230338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.230452 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.230567 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.230693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.230811 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] credential.caching = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.230934 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] auth.external = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.231075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.231201 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.231318 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.231441 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.231559 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.231674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.231788 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.231903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] auth.password = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.232017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.232144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] auth.token = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.232261 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.232375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.232490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.232604 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.232725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.232849 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.232967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.233082 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.233197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.233320 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.233436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.233560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.233676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.233791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.233905 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.234021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] credential.caching = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.234142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.234267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.234392 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.234509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.234624 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] federation.caching = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.234739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.234860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.234992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.235138 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.235254 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.235368 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.235483 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.235598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.235732 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.235850 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.235966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.236081 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.236196 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.236310 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.236433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.236550 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.236671 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.236787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.236902 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.237017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.237143 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.237260 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.237374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.237488 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] identity.caching = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.237602 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.237723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.237848 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] federation.caching = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.237965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.238086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.238209 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.238326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.238440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.238563 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.238679 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.238804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.238919 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.239049 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.239217 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.239419 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.239546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.239663 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.239778 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.239893 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.240008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.240131 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.240249 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.240364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.240478 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.240593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.240716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.240833 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] identity.caching = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.240948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.241062 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.241178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.241295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.241418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.241535 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.241650 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.241777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.241893 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.242007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.242130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.242247 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.242363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.242477 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.242591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.242705 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.242828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.242945 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.243073 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.243201 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.243325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.243444 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.243568 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.243686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.243809 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.243982 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.244106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.244222 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.244346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.244471 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.244618 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.244732 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.244848 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.244962 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.245090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.245090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.245310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.245424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.245540 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.245654 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.245778 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.245894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.246009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.246123 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.246238 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.246352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.246475 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.246593 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.246707 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.246821 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.246936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.247103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.247239 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.247357 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.247473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.247588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.247703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.247818 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.247942 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.248059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.248173 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.248288 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.248402 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.248525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.248651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.248809 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.248927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.249042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.249157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.249279 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.249404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.249521 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.249640 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.249745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.249859 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.249973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.250096 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.250219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.250337 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.250451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.250566 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.250680 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.250804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.250920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.251050 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.251166 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.251281 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.251396 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.251526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.251643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.251759 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.251873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.251987 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.252111 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.252227 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.252342 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.252456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.252571 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.252685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.252810 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.252926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.253041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.253155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.253270 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.253384 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.253507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.253624 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.253738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.253854 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.253969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.254083 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.254206 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.254322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.254436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.254551 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.254665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.254778 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.254902 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.255038 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.255158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.255272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.255387 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.255502 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.255629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.255746 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.255746 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.255963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.256078 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.256193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.256193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.256418 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.256533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.256648 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.256762 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.256876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.257001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.257117 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.257232 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.257346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.257460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.257574 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.257701 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.257818 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.257933 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.258048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.258162 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.258277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.258400 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] resource.caching = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.258516 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.258630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.258744 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.258858 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.258972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.259117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.259236 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.259350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.259464 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.259591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.259706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.259832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.259949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.260063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.260178 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.260293 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.260407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.260530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.260646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.260761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] role.caching = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.260881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.260996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] role.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.261110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.261234 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.261360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.261476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.261591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.261705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.261820 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] resource.caching = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.262007 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.262181 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.262300 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.262415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.262530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.262644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.262767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.262884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.262998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.263207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.263323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.263438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.263566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.263683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.263797 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.263912 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.264025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.264140 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.264263 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.264383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] role.caching = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.264497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.264612 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] role.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.264612 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.264834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.264951 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.265068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.265184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.265299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.265416 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.265539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.265654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.265769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.265882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.265998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.266113 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.266236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.266353 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.266468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.266582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.266696 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.266810 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.266933 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.267062 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.267179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.267293 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.267407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.267522 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.267649 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.267649 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.267867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.267982 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.268135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.268254 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.268378 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.268494 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.268609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.268724 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.268838 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.268952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.269076 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.269192 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] token.caching = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.269307 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.269421 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.269535 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.269650 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.269773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.269889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.270004 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.270118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.270233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.270356 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.270481 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.270597 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.270713 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.270829 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.270944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.271083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.271202 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.271316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.271430 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.271547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.271547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.271771 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.271889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.272003 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.272117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.272231 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.272231 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.272453 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.272453 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.272669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] token.caching = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.272669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.272884 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.272999 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.273121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.273238 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.273352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.connection = **** {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.273476 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.273601 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.273717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.273833 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.273947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.274070 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.274187 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.274302 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.274416 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.274530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.274644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.274767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.274884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.275002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.275259 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.275394 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.275512 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.275636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.275752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.275869 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.275984 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.276098 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.276204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.276326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.276442 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.276556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.276556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.276772 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.276886 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.277010 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.connection = **** {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.277135 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.277261 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.277377 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.277492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.277606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.277721 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.277846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 05:29:33.277965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.277965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.277965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.277965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.277965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.277965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.277965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.277965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.277965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.278886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.278886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.278886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.278886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.278886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.279436 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 21 05:29:33.279436 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80391) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 21 05:29:33.306313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.306580 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.306817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.308993 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.309226 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.309450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.315093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.317754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.342849 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.343093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.343570 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.343801 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.488588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.488820 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.537055 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.537234 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.538271 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.538501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.600935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.601093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.601220 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.601306 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.601400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.601480 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.601543 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.601665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.601783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.601871 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.601936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.602049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.626510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.626736 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.627040 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.627167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.627309 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.627390 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.627454 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.627546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.627634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.627712 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.627857 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.627984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.628121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.628304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.639036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.639334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.651860 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.652064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.652317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.652443 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.652585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.652668 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.652789 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.652883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.652962 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.653037 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.653111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80391) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.653184 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.653392 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.653575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 21 05:29:33.670754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.670956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 05:29:33.671132 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.671239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 05:29:33.671371 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.671498 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 05:29:33.671578 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.671698 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 21 05:29:33.671779 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.671973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.672066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.672229 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.672321 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.672481 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.672580 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.672705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.683590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.683822 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.683948 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.684087 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.684189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.684301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.684382 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.684596 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.692454 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.692647 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.695760 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.695969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.696069 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.696221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.696366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.696507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.696631 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.696839 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.696964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.697173 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.697273 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.697428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.697519 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.697726 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.697852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.698004 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.698089 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.698212 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.698292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.698515 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.698659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.698745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.724849 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.725254 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.726450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.726727 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.726871 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.727142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.732398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.732398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.732607 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.732903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.733176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.733449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.739978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.740202 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.740706 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.740992 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.744865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.745149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.745427 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.745631 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.745748 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.745883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.745963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.746149 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.746276 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.746427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.746512 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.746675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.746801 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.746940 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.747214 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.747485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.765342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.765613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.766251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.766522 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.770159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.771040 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.773800 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.774089 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.774656 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.774949 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.779183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.779984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.782785 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.783594 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.786590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.786890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.787203 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.787345 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.787485 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.787937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.788123 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.788252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.788391 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.788473 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.788772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.789049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.807383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.807754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.807878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.808033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.808240 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.808364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.808496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.808768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.816196 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.816682 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.817908 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.818178 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.818340 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.818630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.821256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.821531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.821655 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.821787 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.821987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.822112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.822269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.822353 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.822503 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.822791 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.835760 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.836037 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.836180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.836320 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.836397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.836589 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.836715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.836838 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.836923 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.837139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.837281 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.837421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.837497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.837807 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.856619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.856880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.859911 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.860432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.863244 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.863757 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.864830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.865340 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.866468 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.866743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.866936 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.867070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.867222 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.867530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.874537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.874973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.876258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.876722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.879509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.879793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.879920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.880051 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.880252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.880394 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.880532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.880771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.889557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.889833 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.889977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.890117 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.890201 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.890493 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.897956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80391) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.898232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.898357 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 21 05:29:33.898488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.898572 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.898763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.898890 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.899068 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.899154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.899327 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.899454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.899595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.899696 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.899818 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.899899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.900102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.900322 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.900413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.900490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.900680 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.900820 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.900913 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.900992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.901190 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.901317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.901400 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.901477 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80391) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.901703 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 21 05:29:33.939458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c4aef169c0 pid: 80391 (default app) Jul 21 05:29:33.939588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c4aef169c0 pid: 80390 (default app) Jul 21 05:29:33.942807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677be0d-fa7d-4e42-b1a6-97bf44eeb238 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:33.943121 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677be0d-fa7d-4e42-b1a6-97bf44eeb238 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:33.943408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677be0d-fa7d-4e42-b1a6-97bf44eeb238 None None] PATH_INFO: `/v3` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:33.947716 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1/1] 10.222.0.44 () {50 vars in 791 bytes} [Thu Jul 21 05:29:33 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:29:41.584362 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3b0df-bd3a-431c-a7af-4f086f6f3fe3 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:41.585251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3b0df-bd3a-431c-a7af-4f086f6f3fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:41.585251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3b0df-bd3a-431c-a7af-4f086f6f3fe3 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:41.590679 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1/2] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:29:41 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:29:41.597309 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f3744-2ee6-49fe-b3dc-db5fa7fc7e51 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:41.598042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f3744-2ee6-49fe-b3dc-db5fa7fc7e51 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:41.598136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f3744-2ee6-49fe-b3dc-db5fa7fc7e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:41.674049 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: INFO dbcounter [None req-432f3744-2ee6-49fe-b3dc-db5fa7fc7e51 None None] Registered counter for database keystone Jul 21 05:29:41.695628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG oslo_db.sqlalchemy.engines [None req-432f3744-2ee6-49fe-b3dc-db5fa7fc7e51 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80391) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 05:29:41.707254 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writer thread running {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 05:29:41.892228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG passlib.handlers.bcrypt [None req-432f3744-2ee6-49fe-b3dc-db5fa7fc7e51 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80391) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 05:29:41.892560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG passlib.handlers.bcrypt [None req-432f3744-2ee6-49fe-b3dc-db5fa7fc7e51 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80391) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 05:29:41.947306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-432f3744-2ee6-49fe-b3dc-db5fa7fc7e51 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:29:41.948906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-432f3744-2ee6-49fe-b3dc-db5fa7fc7e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:29:42.076967 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 2/3] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:29:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 21 05:29:42.312700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3b0df-bd3a-431c-a7af-4f086f6f3fe3 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:42.396495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: INFO dbcounter [None req-33b3b0df-bd3a-431c-a7af-4f086f6f3fe3 None None] Registered counter for database keystone Jul 21 05:29:42.418650 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG oslo_db.sqlalchemy.engines [None req-33b3b0df-bd3a-431c-a7af-4f086f6f3fe3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80390) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 21 05:29:42.429354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writer thread running {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 21 05:29:42.553967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3b0df-bd3a-431c-a7af-4f086f6f3fe3 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:42.559559 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda2158-9237-4c28-b358-b5b915da2d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:42.560227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda2158-9237-4c28-b358-b5b915da2d25 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:42.560513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda2158-9237-4c28-b358-b5b915da2d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:42.560784 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda2158-9237-4c28-b358-b5b915da2d25 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:42.569944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda2158-9237-4c28-b358-b5b915da2d25 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:42.621706 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0fda2158-9237-4c28-b358-b5b915da2d25 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 05:29:42.622279 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda2158-9237-4c28-b358-b5b915da2d25 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:42.632274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 2/4] 10.222.0.44 () {58 vars in 1324 bytes} [Thu Jul 21 05:29:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 05:29:42.639561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f3744-2ee6-49fe-b3dc-db5fa7fc7e51 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:42.665540 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f3744-2ee6-49fe-b3dc-db5fa7fc7e51 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:42.671718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b7c03-abd9-4125-92aa-791dcf47d88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:42.672893 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b7c03-abd9-4125-92aa-791dcf47d88a admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:42.673189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b7c03-abd9-4125-92aa-791dcf47d88a admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:42.673462 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b7c03-abd9-4125-92aa-791dcf47d88a admin admin] PATH_INFO: `/v3/projects/f7ec186b03924ec6a68eab5b0ea7d82d` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:42.677882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b7c03-abd9-4125-92aa-791dcf47d88a admin admin] RBAC: Authorizing `identity:get_project(project_id=f7ec186b03924ec6a68eab5b0ea7d82d)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:42.729623 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: WARNING keystone.common.rbac_enforcer.enforcer [None req-756b7c03-abd9-4125-92aa-791dcf47d88a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 21 05:29:42.729965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b7c03-abd9-4125-92aa-791dcf47d88a admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:42.734646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 3/5] 10.222.0.44 () {56 vars in 1207 bytes} [Thu Jul 21 05:29:42 2022] GET /identity/v3/projects/f7ec186b03924ec6a68eab5b0ea7d82d => generated 344 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:29:44.764393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed698e7f-2e61-447a-8e55-3f79adce94b6 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:44.765219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed698e7f-2e61-447a-8e55-3f79adce94b6 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:44.765219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed698e7f-2e61-447a-8e55-3f79adce94b6 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:44.772002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 3/6] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:29:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:29:44.780133 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4c2c0-204e-4b25-83fb-d2054191c1aa None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:44.780583 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4c2c0-204e-4b25-83fb-d2054191c1aa None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:44.780675 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4c2c0-204e-4b25-83fb-d2054191c1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:44.801479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-8bb4c2c0-204e-4b25-83fb-d2054191c1aa None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:29:44.802992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8bb4c2c0-204e-4b25-83fb-d2054191c1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:29:44.811697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 4/7] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:29:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 21 05:29:44.834805 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed698e7f-2e61-447a-8e55-3f79adce94b6 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:44.855644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed698e7f-2e61-447a-8e55-3f79adce94b6 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:44.860879 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c9863-8ea0-4ae1-bf2a-2de26956f5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:44.861508 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c9863-8ea0-4ae1-bf2a-2de26956f5ed admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:44.861796 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c9863-8ea0-4ae1-bf2a-2de26956f5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:44.862071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c9863-8ea0-4ae1-bf2a-2de26956f5ed admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:44.870486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c9863-8ea0-4ae1-bf2a-2de26956f5ed admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:44.872316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c9863-8ea0-4ae1-bf2a-2de26956f5ed admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:44.881702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 4/8] 10.222.0.44 () {58 vars in 1324 bytes} [Thu Jul 21 05:29:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 05:29:44.890610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4c2c0-204e-4b25-83fb-d2054191c1aa None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:44.904062 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4c2c0-204e-4b25-83fb-d2054191c1aa None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:44.909898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374137d2-6df7-4a81-9688-36ea8835dac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:44.910768 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374137d2-6df7-4a81-9688-36ea8835dac8 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:44.911062 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374137d2-6df7-4a81-9688-36ea8835dac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:44.911420 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374137d2-6df7-4a81-9688-36ea8835dac8 admin admin] PATH_INFO: `/v3/users/3b3cc4a53587419894e0f708d1e60e89` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:44.913263 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374137d2-6df7-4a81-9688-36ea8835dac8 admin admin] RBAC: Authorizing `identity:get_user(user_id=3b3cc4a53587419894e0f708d1e60e89)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:44.915155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374137d2-6df7-4a81-9688-36ea8835dac8 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:44.916840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 5/9] 10.222.0.44 () {56 vars in 1198 bytes} [Thu Jul 21 05:29:44 2022] GET /identity/v3/users/3b3cc4a53587419894e0f708d1e60e89 => generated 247 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:29:47.001706 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f552dfc-931c-4c7a-ae1e-eeeaaba845c9 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:47.002234 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f552dfc-931c-4c7a-ae1e-eeeaaba845c9 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:47.002234 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f552dfc-931c-4c7a-ae1e-eeeaaba845c9 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:47.003575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 5/10] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:29:47.009014 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921852f-a367-4f63-9764-808a20fdded0 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:47.009569 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921852f-a367-4f63-9764-808a20fdded0 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:47.009661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921852f-a367-4f63-9764-808a20fdded0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:47.010478 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac8329-1810-44bd-9f3d-687b13680056 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:47.010769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac8329-1810-44bd-9f3d-687b13680056 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:47.011046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac8329-1810-44bd-9f3d-687b13680056 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:47.012317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 6/11] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:29:47.017229 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e3c13-f174-4546-8b97-84e958034820 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:47.017512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e3c13-f174-4546-8b97-84e958034820 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:47.017785 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e3c13-f174-4546-8b97-84e958034820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:47.027002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-b921852f-a367-4f63-9764-808a20fdded0 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:29:47.028228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b921852f-a367-4f63-9764-808a20fdded0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:29:47.061113 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG passlib.handlers.bcrypt [None req-b49e3c13-f174-4546-8b97-84e958034820 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80390) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 21 05:29:47.061819 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG passlib.handlers.bcrypt [None req-b49e3c13-f174-4546-8b97-84e958034820 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80390) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 21 05:29:47.107494 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-b49e3c13-f174-4546-8b97-84e958034820 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:29:47.109120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b49e3c13-f174-4546-8b97-84e958034820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:29:47.120536 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 6/12] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:29:47.142479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921852f-a367-4f63-9764-808a20fdded0 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:47.168153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:47.208342 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 7/13] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:29:47.230950 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e3c13-f174-4546-8b97-84e958034820 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:47.249426 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921852f-a367-4f63-9764-808a20fdded0 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:47.282419 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71502f-b067-4b03-9c6e-0e98ebdad485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:47.283031 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71502f-b067-4b03-9c6e-0e98ebdad485 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:47.283357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71502f-b067-4b03-9c6e-0e98ebdad485 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:47.283632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71502f-b067-4b03-9c6e-0e98ebdad485 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:47.330073 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e3c13-f174-4546-8b97-84e958034820 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:47.332423 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:47.350325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71502f-b067-4b03-9c6e-0e98ebdad485 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:47.351867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71502f-b067-4b03-9c6e-0e98ebdad485 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:47.368507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d7800-bd0b-4ecd-b3dc-7e536cc8c2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:47.369628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d7800-bd0b-4ecd-b3dc-7e536cc8c2cc None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:47.369937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d7800-bd0b-4ecd-b3dc-7e536cc8c2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:47.370235 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d7800-bd0b-4ecd-b3dc-7e536cc8c2cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:47.375487 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:47.382240 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d7800-bd0b-4ecd-b3dc-7e536cc8c2cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:29:47.393509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-950d7800-bd0b-4ecd-b3dc-7e536cc8c2cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:29:47.393509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:29:47.393509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:29:47.393509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:29:47.393509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:29:47.393509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:29:47.393509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:29:47.393509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:29:47.393509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:29:47.393509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:29:47.393509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:29:47.393509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:29:47.393509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:29:47.393509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:29:47.393509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:29:47.393509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:29:47.394637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:29:47.394637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:29:47.394637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:29:47.394637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:29:47.394637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:29:47.394637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:29:47.394637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:29:47.394637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:29:47.394637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:29:47.394637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:29:47.394637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:29:47.394637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:29:47.394637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:29:47.394637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:29:47.394637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:29:47.394637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:29:47.395605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:29:47.395605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:29:47.395605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:29:47.395605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:29:47.395605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:29:47.395605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:29:47.395605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:29:47.395605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:29:47.395605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:29:47.395605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:29:47.395605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:29:47.395605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:29:47.395605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 8/14] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:29:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:29:47.401870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d7800-bd0b-4ecd-b3dc-7e536cc8c2cc None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:47.465606 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 7/15] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:29:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 05:29:47.472010 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71502f-b067-4b03-9c6e-0e98ebdad485 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:47.512104 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d7800-bd0b-4ecd-b3dc-7e536cc8c2cc None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:47.551622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f8c6b-578e-4dca-b198-bd728af57a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:47.552216 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f8c6b-578e-4dca-b198-bd728af57a5c None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:47.552504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f8c6b-578e-4dca-b198-bd728af57a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:47.552751 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f8c6b-578e-4dca-b198-bd728af57a5c None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:47.553534 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f8c6b-578e-4dca-b198-bd728af57a5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:29:47.553843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f8c6b-578e-4dca-b198-bd728af57a5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:47.555358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f8c6b-578e-4dca-b198-bd728af57a5c None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:47.574690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 9/16] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:29:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:29:47.574876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71502f-b067-4b03-9c6e-0e98ebdad485 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:47.575863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f8c6b-578e-4dca-b198-bd728af57a5c None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:47.597257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:47.604710 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:47.609956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f592ab6-ac2f-480f-999c-9eb7e50462ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:47.610782 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f592ab6-ac2f-480f-999c-9eb7e50462ab None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:47.611101 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f592ab6-ac2f-480f-999c-9eb7e50462ab None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:47.611381 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f592ab6-ac2f-480f-999c-9eb7e50462ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:47.621422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f592ab6-ac2f-480f-999c-9eb7e50462ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:47.622916 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f592ab6-ac2f-480f-999c-9eb7e50462ab None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:47.643807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-8f592ab6-ac2f-480f-999c-9eb7e50462ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:29:47.643807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:29:47.643807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:29:47.643807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:29:47.643807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:29:47.643807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:29:47.643807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:29:47.643807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:29:47.643807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:29:47.643807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:29:47.643807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:29:47.643807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:29:47.643807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:29:47.643807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:29:47.643807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:29:47.644738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:29:47.644738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:29:47.644738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:29:47.644738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:29:47.644738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:29:47.644738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:29:47.644738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:29:47.644738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:29:47.644738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:29:47.644738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:29:47.644738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:29:47.644738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:29:47.644738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:29:47.644738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:29:47.644738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:29:47.644738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:29:47.644738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:29:47.645687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:29:47.645687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:29:47.645687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:29:47.645687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:29:47.647246 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 8/17] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:29:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:29:47.653219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f592ab6-ac2f-480f-999c-9eb7e50462ab None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:47.683131 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f8c6b-578e-4dca-b198-bd728af57a5c None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:47.717892 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ae9af-2beb-4a61-b653-98eb0019cc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:47.718777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ae9af-2beb-4a61-b653-98eb0019cc49 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:47.719087 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ae9af-2beb-4a61-b653-98eb0019cc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:47.719360 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ae9af-2beb-4a61-b653-98eb0019cc49 None admin] PATH_INFO: `/v3/users/3b3cc4a53587419894e0f708d1e60e89` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:47.720929 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ae9af-2beb-4a61-b653-98eb0019cc49 None admin] RBAC: Authorizing `identity:get_user(user_id=3b3cc4a53587419894e0f708d1e60e89)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:47.722406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987ae9af-2beb-4a61-b653-98eb0019cc49 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:47.724071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 10/18] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:29:47 2022] GET /identity/v3/users/3b3cc4a53587419894e0f708d1e60e89 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:29:47.729560 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ae9af-2beb-4a61-b653-98eb0019cc49 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:47.752845 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f592ab6-ac2f-480f-999c-9eb7e50462ab None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:47.789312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3425ff-38c2-4d18-bda4-10ded441847b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:47.789974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3425ff-38c2-4d18-bda4-10ded441847b None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:47.790246 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3425ff-38c2-4d18-bda4-10ded441847b None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:47.790517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3425ff-38c2-4d18-bda4-10ded441847b None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:47.791377 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3425ff-38c2-4d18-bda4-10ded441847b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:29:47.791681 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3425ff-38c2-4d18-bda4-10ded441847b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:47.793081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3425ff-38c2-4d18-bda4-10ded441847b None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:47.810449 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 9/19] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:29:47.823246 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:47.831167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ae9af-2beb-4a61-b653-98eb0019cc49 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:47.867935 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8cd7b-801d-41a0-b09a-06c256f26b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:47.868837 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8cd7b-801d-41a0-b09a-06c256f26b2a None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:47.869122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8cd7b-801d-41a0-b09a-06c256f26b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:47.869398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8cd7b-801d-41a0-b09a-06c256f26b2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:47.870659 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8cd7b-801d-41a0-b09a-06c256f26b2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:47.872182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8cd7b-801d-41a0-b09a-06c256f26b2a None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:47.873826 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 11/20] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:29:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:29:47.880224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3425ff-38c2-4d18-bda4-10ded441847b None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:47.918238 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:47.988854 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3425ff-38c2-4d18-bda4-10ded441847b None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:48.023055 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd134b2-e9c0-4cd2-ac46-b7db8a159ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:48.023716 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd134b2-e9c0-4cd2-ac46-b7db8a159ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:48.024022 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd134b2-e9c0-4cd2-ac46-b7db8a159ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:48.024281 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd134b2-e9c0-4cd2-ac46-b7db8a159ba3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:48.025242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd134b2-e9c0-4cd2-ac46-b7db8a159ba3 None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.domain.id=default, user.id=3b3cc4a53587419894e0f708d1e60e89) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:29:48.025529 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd134b2-e9c0-4cd2-ac46-b7db8a159ba3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:48.026996 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd134b2-e9c0-4cd2-ac46-b7db8a159ba3 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:48.034275 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 10/21] 10.222.0.44 () {56 vars in 1322 bytes} [Thu Jul 21 05:29:47 2022] GET /identity/v3/role_assignments?user.id=3b3cc4a53587419894e0f708d1e60e89&scope.domain.id=default&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:29:50.095771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59f34e-397e-49ce-aa47-e6ed96cbab93 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:50.096602 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59f34e-397e-49ce-aa47-e6ed96cbab93 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:50.097116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59f34e-397e-49ce-aa47-e6ed96cbab93 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:50.098644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 12/22] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:29:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:29:50.104240 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af2860-cc61-4de5-a731-fedac0784415 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:50.104657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af2860-cc61-4de5-a731-fedac0784415 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:50.104793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af2860-cc61-4de5-a731-fedac0784415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:50.128011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-13af2860-cc61-4de5-a731-fedac0784415 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:29:50.129229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-13af2860-cc61-4de5-a731-fedac0784415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:29:50.244497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 11/23] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:29:50.279915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59f34e-397e-49ce-aa47-e6ed96cbab93 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:50.471199 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59f34e-397e-49ce-aa47-e6ed96cbab93 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:50.528983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8a365-a384-406e-8770-00762bda57cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:50.531694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8a365-a384-406e-8770-00762bda57cf None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:50.532127 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8a365-a384-406e-8770-00762bda57cf None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:50.532538 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8a365-a384-406e-8770-00762bda57cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:50.562262 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b8a365-a384-406e-8770-00762bda57cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:50.564079 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b8a365-a384-406e-8770-00762bda57cf None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:50.583738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-b3b8a365-a384-406e-8770-00762bda57cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:29:50.583738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:29:50.583738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:29:50.583738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:29:50.583738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:29:50.583738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:29:50.583738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:29:50.583738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:29:50.583738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:29:50.583738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:29:50.583738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:29:50.583738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:29:50.583738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:29:50.583738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:29:50.583738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:29:50.584766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:29:50.584766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:29:50.584766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:29:50.584766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:29:50.584766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:29:50.584766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:29:50.584766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:29:50.584766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:29:50.584766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:29:50.584766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:29:50.584766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:29:50.584766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:29:50.584766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:29:50.584766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:29:50.584766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:29:50.584766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:29:50.584766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:29:50.585784 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:29:50.585784 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:29:50.585784 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:29:50.585784 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:29:50.587584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 13/24] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:29:50.596152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af2860-cc61-4de5-a731-fedac0784415 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:50.656601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:50.704518 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa25c1-6aac-4124-af7d-650cdd9b3293 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:50.705150 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa25c1-6aac-4124-af7d-650cdd9b3293 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:50.711909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa25c1-6aac-4124-af7d-650cdd9b3293 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:50.713611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 14/25] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:29:50 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:29:50.719412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2867f-a37c-4633-8273-4a0a808b3512 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:50.719988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2867f-a37c-4633-8273-4a0a808b3512 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:50.720455 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2867f-a37c-4633-8273-4a0a808b3512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:50.746755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-19b2867f-a37c-4633-8273-4a0a808b3512 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:29:50.748296 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-19b2867f-a37c-4633-8273-4a0a808b3512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:29:50.759692 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:50.778758 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af2860-cc61-4de5-a731-fedac0784415 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:50.826938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59299ddb-9e9d-40e1-a5f0-07fbeb1f2a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:50.827891 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59299ddb-9e9d-40e1-a5f0-07fbeb1f2a1e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:50.828350 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59299ddb-9e9d-40e1-a5f0-07fbeb1f2a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:50.828768 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59299ddb-9e9d-40e1-a5f0-07fbeb1f2a1e None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:50.829639 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59299ddb-9e9d-40e1-a5f0-07fbeb1f2a1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:29:50.830089 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59299ddb-9e9d-40e1-a5f0-07fbeb1f2a1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:50.831680 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59299ddb-9e9d-40e1-a5f0-07fbeb1f2a1e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:50.843489 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 12/26] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:29:50.845172 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833bd80-432b-41d7-ba27-191e4e79d5a4 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:50.845613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833bd80-432b-41d7-ba27-191e4e79d5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:50.846029 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e833bd80-432b-41d7-ba27-191e4e79d5a4 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:50.847800 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 13/27] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:29:50.849216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833bd80-432b-41d7-ba27-191e4e79d5a4 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:50.917364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 15/28] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:29:50.918983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0eacb3-524b-4ff0-9573-4868bb07bd47 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:50.919395 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0eacb3-524b-4ff0-9573-4868bb07bd47 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:50.919770 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0eacb3-524b-4ff0-9573-4868bb07bd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:50.945240 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-0f0eacb3-524b-4ff0-9573-4868bb07bd47 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:29:50.946674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-0f0eacb3-524b-4ff0-9573-4868bb07bd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:29:50.950031 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:50.975706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e833bd80-432b-41d7-ba27-191e4e79d5a4 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:51.022574 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e30cd0-8e86-43be-a3b9-1ec8b66ecb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:51.023621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e30cd0-8e86-43be-a3b9-1ec8b66ecb5b None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:51.024067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e30cd0-8e86-43be-a3b9-1ec8b66ecb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:51.024581 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e30cd0-8e86-43be-a3b9-1ec8b66ecb5b None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:51.025689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e30cd0-8e86-43be-a3b9-1ec8b66ecb5b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:51.027292 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 16/29] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:29:51.027479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e30cd0-8e86-43be-a3b9-1ec8b66ecb5b None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:51.029222 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75517439-9dd0-440b-9765-8ae1893cbdd9 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:51.029736 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75517439-9dd0-440b-9765-8ae1893cbdd9 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:51.030307 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75517439-9dd0-440b-9765-8ae1893cbdd9 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:51.031872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 17/30] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:29:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:29:51.032867 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75517439-9dd0-440b-9765-8ae1893cbdd9 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:51.123327 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:51.152851 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75517439-9dd0-440b-9765-8ae1893cbdd9 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:51.173146 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 14/31] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jul 21 05:29:50 2022] POST /identity/v3/projects => generated 301 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:29:51.175334 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1ad67-3662-4adf-9c5a-00fa09be888e None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:51.175827 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1ad67-3662-4adf-9c5a-00fa09be888e None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:51.176313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1ad67-3662-4adf-9c5a-00fa09be888e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:51.193333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c55cf-584d-4e0e-8bca-4e5be3e5472e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:51.194181 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c55cf-584d-4e0e-8bca-4e5be3e5472e None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:51.194468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c55cf-584d-4e0e-8bca-4e5be3e5472e None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:51.194835 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c55cf-584d-4e0e-8bca-4e5be3e5472e None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:51.195635 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c55cf-584d-4e0e-8bca-4e5be3e5472e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:51.195739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-bef1ad67-3662-4adf-9c5a-00fa09be888e None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:29:51.197419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c55cf-584d-4e0e-8bca-4e5be3e5472e None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:51.197775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-bef1ad67-3662-4adf-9c5a-00fa09be888e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:29:51.345696 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 15/32] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:29:51.346533 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1ad67-3662-4adf-9c5a-00fa09be888e None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:51.383840 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 18/33] 10.222.0.44 () {60 vars in 1131 bytes} [Thu Jul 21 05:29:51 2022] POST /identity/v3/roles => generated 225 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:29:51.391342 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c55cf-584d-4e0e-8bca-4e5be3e5472e None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:51.473467 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:51.538300 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1ad67-3662-4adf-9c5a-00fa09be888e None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:51.563225 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c55cf-584d-4e0e-8bca-4e5be3e5472e None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:51.599117 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bfb74-5cc9-4dee-89dd-bdb0a2a5ca6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:51.600156 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bfb74-5cc9-4dee-89dd-bdb0a2a5ca6a None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:51.600659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bfb74-5cc9-4dee-89dd-bdb0a2a5ca6a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:51.601040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bfb74-5cc9-4dee-89dd-bdb0a2a5ca6a None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:51.601902 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bfb74-5cc9-4dee-89dd-bdb0a2a5ca6a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:51.604454 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bfb74-5cc9-4dee-89dd-bdb0a2a5ca6a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:51.610806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4149c03-c421-4770-94f8-986e9fe4a6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:51.611941 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4149c03-c421-4770-94f8-986e9fe4a6fd None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:51.612394 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4149c03-c421-4770-94f8-986e9fe4a6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:51.613015 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4149c03-c421-4770-94f8-986e9fe4a6fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:51.615425 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:51.624842 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 16/34] 10.222.0.44 () {60 vars in 1131 bytes} [Thu Jul 21 05:29:51 2022] POST /identity/v3/roles => generated 219 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:29:51.629895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4149c03-c421-4770-94f8-986e9fe4a6fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:29:51.639487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-a4149c03-c421-4770-94f8-986e9fe4a6fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:29:51.639487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:29:51.639487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:29:51.639487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:29:51.639487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:29:51.639487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:29:51.639487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:29:51.639487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:29:51.639487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:29:51.639487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:29:51.639487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:29:51.639487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:29:51.639487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:29:51.639487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:29:51.639487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:29:51.639487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:29:51.640489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:29:51.640489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:29:51.640489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:29:51.640489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:29:51.640489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:29:51.640489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:29:51.640489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:29:51.640489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:29:51.640489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:29:51.640489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:29:51.640489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:29:51.640489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:29:51.640489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:29:51.640489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:29:51.640489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:29:51.640489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:29:51.641265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:29:51.641265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:29:51.641265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:29:51.641265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:29:51.641265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:29:51.641265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:29:51.641265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:29:51.641265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:29:51.641265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:29:51.641265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:29:51.641265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:29:51.641265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:29:51.651817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 19/35] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:29:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:29:51.652437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bfb74-5cc9-4dee-89dd-bdb0a2a5ca6a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:51.682601 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb259d-896d-45f3-a92f-15fc22524211 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:51.683311 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb259d-896d-45f3-a92f-15fc22524211 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:51.683781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb259d-896d-45f3-a92f-15fc22524211 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:51.687613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 20/36] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:29:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:29:51.691495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59310a6-a1f6-43d9-b100-0fb37f681e5a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:51.691973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59310a6-a1f6-43d9-b100-0fb37f681e5a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:51.692415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59310a6-a1f6-43d9-b100-0fb37f681e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:51.722490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-f59310a6-a1f6-43d9-b100-0fb37f681e5a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:29:51.727689 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f59310a6-a1f6-43d9-b100-0fb37f681e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:29:51.813049 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bfb74-5cc9-4dee-89dd-bdb0a2a5ca6a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:51.820552 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 21/37] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:29:51.843536 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59310a6-a1f6-43d9-b100-0fb37f681e5a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:51.847604 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:51.858488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba778bb-b28b-49b5-baed-4bd082ae70d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:51.859489 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba778bb-b28b-49b5-baed-4bd082ae70d9 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:51.859804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba778bb-b28b-49b5-baed-4bd082ae70d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:51.860884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba778bb-b28b-49b5-baed-4bd082ae70d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:51.860884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba778bb-b28b-49b5-baed-4bd082ae70d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:29:51.862127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba778bb-b28b-49b5-baed-4bd082ae70d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:51.863613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba778bb-b28b-49b5-baed-4bd082ae70d9 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:51.876332 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 17/38] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:29:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:29:51.882508 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba778bb-b28b-49b5-baed-4bd082ae70d9 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:51.964419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59310a6-a1f6-43d9-b100-0fb37f681e5a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:51.984359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba778bb-b28b-49b5-baed-4bd082ae70d9 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:51.994337 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:52.005964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b36fd7-cc7d-4288-ac31-410a9f808aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:52.006649 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b36fd7-cc7d-4288-ac31-410a9f808aef None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:52.006929 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b36fd7-cc7d-4288-ac31-410a9f808aef None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:52.007236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b36fd7-cc7d-4288-ac31-410a9f808aef None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:52.007996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b36fd7-cc7d-4288-ac31-410a9f808aef None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:52.009560 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b36fd7-cc7d-4288-ac31-410a9f808aef None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:52.020064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a65514-007f-402d-aeb4-4e43551df9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:52.020968 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a65514-007f-402d-aeb4-4e43551df9ed None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:52.021258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a65514-007f-402d-aeb4-4e43551df9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:52.021537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a65514-007f-402d-aeb4-4e43551df9ed None admin] PATH_INFO: `/v3/users/3b3cc4a53587419894e0f708d1e60e89` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:52.022750 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a65514-007f-402d-aeb4-4e43551df9ed None admin] RBAC: Authorizing `identity:get_user(user_id=3b3cc4a53587419894e0f708d1e60e89)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:52.023542 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 22/39] 10.222.0.44 () {60 vars in 1131 bytes} [Thu Jul 21 05:29:51 2022] POST /identity/v3/roles => generated 223 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:29:52.024307 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a65514-007f-402d-aeb4-4e43551df9ed None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:52.026247 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 18/40] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:29:51 2022] GET /identity/v3/users/3b3cc4a53587419894e0f708d1e60e89 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:29:52.031968 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b36fd7-cc7d-4288-ac31-410a9f808aef None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:52.062004 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f48798-0704-4438-bced-ff83b30be72c None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:52.062330 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f48798-0704-4438-bced-ff83b30be72c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:52.062571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f48798-0704-4438-bced-ff83b30be72c None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:52.064112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 19/41] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:29:52.069799 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df862f-141c-40c4-b41e-7964934a0f95 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:52.070148 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df862f-141c-40c4-b41e-7964934a0f95 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:52.070374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df862f-141c-40c4-b41e-7964934a0f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:52.089228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-69df862f-141c-40c4-b41e-7964934a0f95 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:29:52.090473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-69df862f-141c-40c4-b41e-7964934a0f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:29:52.152419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b36fd7-cc7d-4288-ac31-410a9f808aef None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:52.196490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b2010-cd6e-4b28-bd94-fbd5f6a2045c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:52.197776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b2010-cd6e-4b28-bd94-fbd5f6a2045c None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:52.198061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b2010-cd6e-4b28-bd94-fbd5f6a2045c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:52.198341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b2010-cd6e-4b28-bd94-fbd5f6a2045c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:52.199640 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b2010-cd6e-4b28-bd94-fbd5f6a2045c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:52.201123 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b2010-cd6e-4b28-bd94-fbd5f6a2045c None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:52.202823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 23/42] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:29:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:29:52.208949 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b2010-cd6e-4b28-bd94-fbd5f6a2045c None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:52.265593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:52.277883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 20/43] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:29:52.278754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:52.301669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df862f-141c-40c4-b41e-7964934a0f95 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:52.328006 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b2010-cd6e-4b28-bd94-fbd5f6a2045c None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:52.361808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d7291-2ab2-4417-8001-676f7e18ea91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:52.362681 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d7291-2ab2-4417-8001-676f7e18ea91 None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:52.362967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d7291-2ab2-4417-8001-676f7e18ea91 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:52.363274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d7291-2ab2-4417-8001-676f7e18ea91 None admin] PATH_INFO: `/v3/domains/default/users/3b3cc4a53587419894e0f708d1e60e89/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:52.365346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d7291-2ab2-4417-8001-676f7e18ea91 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3b3cc4a53587419894e0f708d1e60e89, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:52.366807 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d7291-2ab2-4417-8001-676f7e18ea91 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:52.382026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 24/44] 10.222.0.44 () {58 vars in 1361 bytes} [Thu Jul 21 05:29:52 2022] PUT /identity/v3/domains/default/users/3b3cc4a53587419894e0f708d1e60e89/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:29:52.401041 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df862f-141c-40c4-b41e-7964934a0f95 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:52.439266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5e426-754e-4c82-b7ee-c6b676fd98c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:52.440111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5e426-754e-4c82-b7ee-c6b676fd98c3 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:52.440399 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5e426-754e-4c82-b7ee-c6b676fd98c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:52.440675 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5e426-754e-4c82-b7ee-c6b676fd98c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:52.441983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5e426-754e-4c82-b7ee-c6b676fd98c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:52.443549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5e426-754e-4c82-b7ee-c6b676fd98c3 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:52.445241 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 21/45] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:29:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:29:52.451308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d7291-2ab2-4417-8001-676f7e18ea91 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:52.559845 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d7291-2ab2-4417-8001-676f7e18ea91 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:52.572519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:52.594861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155dd19-0092-422b-a3d6-bb3930ff1ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:52.595593 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155dd19-0092-422b-a3d6-bb3930ff1ab3 None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:52.595877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155dd19-0092-422b-a3d6-bb3930ff1ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:52.596152 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155dd19-0092-422b-a3d6-bb3930ff1ab3 None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:52.596880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155dd19-0092-422b-a3d6-bb3930ff1ab3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:52.598345 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155dd19-0092-422b-a3d6-bb3930ff1ab3 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:52.625949 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 25/46] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jul 21 05:29:52 2022] POST /identity/v3/projects => generated 312 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:29:54.390024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e53971-e297-4506-8b93-c69c5ff6aa25 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:54.390568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e53971-e297-4506-8b93-c69c5ff6aa25 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:54.390568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e53971-e297-4506-8b93-c69c5ff6aa25 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:54.391964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 22/47] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:29:54.397354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b0e84-6f6b-4b3d-9652-76a37ec02f07 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:54.397761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b0e84-6f6b-4b3d-9652-76a37ec02f07 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:54.397945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b0e84-6f6b-4b3d-9652-76a37ec02f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:54.416018 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-f45b0e84-6f6b-4b3d-9652-76a37ec02f07 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:29:54.417221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f45b0e84-6f6b-4b3d-9652-76a37ec02f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:29:54.487746 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 26/48] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:29:54.510374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e53971-e297-4506-8b93-c69c5ff6aa25 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:54.570859 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:54.616308 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e53971-e297-4506-8b93-c69c5ff6aa25 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:54.618705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21094199-a339-4cdb-b614-1b78eed4db31 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:54.618995 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21094199-a339-4cdb-b614-1b78eed4db31 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:54.619314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21094199-a339-4cdb-b614-1b78eed4db31 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:54.620707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 27/49] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:29:54.625656 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ffc14-fe5e-4ccb-b329-b93a114ae15a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:54.625946 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ffc14-fe5e-4ccb-b329-b93a114ae15a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:54.626223 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ffc14-fe5e-4ccb-b329-b93a114ae15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:54.643506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-868ffc14-fe5e-4ccb-b329-b93a114ae15a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:29:54.644634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-868ffc14-fe5e-4ccb-b329-b93a114ae15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:29:54.652166 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7146d6b-6c41-49b2-9ebb-6b441803e097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:54.652978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7146d6b-6c41-49b2-9ebb-6b441803e097 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:54.653260 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7146d6b-6c41-49b2-9ebb-6b441803e097 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:54.653531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7146d6b-6c41-49b2-9ebb-6b441803e097 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:54.668323 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7146d6b-6c41-49b2-9ebb-6b441803e097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:29:54.677118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-a7146d6b-6c41-49b2-9ebb-6b441803e097 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:29:54.677118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:29:54.677118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:29:54.677118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:29:54.677118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:29:54.677118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:29:54.677118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:29:54.677118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:29:54.677118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:29:54.677118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:29:54.677118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:29:54.677118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:29:54.677118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:29:54.677118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:29:54.677118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:29:54.677118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:29:54.678009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:29:54.678009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:29:54.678009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:29:54.678009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:29:54.678009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:29:54.678009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:29:54.678009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:29:54.678009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:29:54.678009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:29:54.678009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:29:54.678009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:29:54.678009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:29:54.678009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:29:54.678009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:29:54.678009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:29:54.678009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:29:54.679094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:29:54.679094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:29:54.679094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:29:54.679094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:29:54.679094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:29:54.679094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:29:54.679094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:29:54.679094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:29:54.679094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:29:54.679094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:29:54.679094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:29:54.679094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:29:54.679094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 23/50] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:29:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:29:54.684992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7146d6b-6c41-49b2-9ebb-6b441803e097 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:54.719044 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 28/51] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:29:54.738332 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ffc14-fe5e-4ccb-b329-b93a114ae15a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:54.762936 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:54.786945 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7146d6b-6c41-49b2-9ebb-6b441803e097 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:54.803240 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:54.819751 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f027478-0d3b-4a5b-83e5-294f60151f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:54.820425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f027478-0d3b-4a5b-83e5-294f60151f2a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:54.820707 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f027478-0d3b-4a5b-83e5-294f60151f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:54.820985 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f027478-0d3b-4a5b-83e5-294f60151f2a None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:54.821729 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f027478-0d3b-4a5b-83e5-294f60151f2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:29:54.822039 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f027478-0d3b-4a5b-83e5-294f60151f2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:54.823472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f027478-0d3b-4a5b-83e5-294f60151f2a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:54.831881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 24/52] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:29:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:29:54.837280 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f027478-0d3b-4a5b-83e5-294f60151f2a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:54.841620 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ffc14-fe5e-4ccb-b329-b93a114ae15a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:54.941331 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f027478-0d3b-4a5b-83e5-294f60151f2a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:54.977137 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1baff-7dd9-4e03-9dc2-05eb043934e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:54.978043 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1baff-7dd9-4e03-9dc2-05eb043934e6 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:54.978327 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1baff-7dd9-4e03-9dc2-05eb043934e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:54.978613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1baff-7dd9-4e03-9dc2-05eb043934e6 None admin] PATH_INFO: `/v3/users/3b3cc4a53587419894e0f708d1e60e89` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:54.979849 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1baff-7dd9-4e03-9dc2-05eb043934e6 None admin] RBAC: Authorizing `identity:get_user(user_id=3b3cc4a53587419894e0f708d1e60e89)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:54.981335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1baff-7dd9-4e03-9dc2-05eb043934e6 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:54.982979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 25/53] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:29:54 2022] GET /identity/v3/users/3b3cc4a53587419894e0f708d1e60e89 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:29:54.986609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3dee2-9942-4f09-9187-e0213f9dab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:54.987491 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3dee2-9942-4f09-9187-e0213f9dab12 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:54.987782 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3dee2-9942-4f09-9187-e0213f9dab12 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:54.988060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3dee2-9942-4f09-9187-e0213f9dab12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:54.988358 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1baff-7dd9-4e03-9dc2-05eb043934e6 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:54.989244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3dee2-9942-4f09-9187-e0213f9dab12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:54.990698 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3dee2-9942-4f09-9187-e0213f9dab12 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:54.992377 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 29/54] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:29:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:29:54.997681 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3dee2-9942-4f09-9187-e0213f9dab12 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:55.035668 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:55.085306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1baff-7dd9-4e03-9dc2-05eb043934e6 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:55.086889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:55.097460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3dee2-9942-4f09-9187-e0213f9dab12 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:55.118980 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56fc4c-6290-438f-a144-f02429832216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:55.119795 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56fc4c-6290-438f-a144-f02429832216 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:55.120076 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56fc4c-6290-438f-a144-f02429832216 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:55.120352 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56fc4c-6290-438f-a144-f02429832216 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:55.121514 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56fc4c-6290-438f-a144-f02429832216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:55.122972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56fc4c-6290-438f-a144-f02429832216 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:55.124608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 26/55] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:29:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:29:55.130331 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56fc4c-6290-438f-a144-f02429832216 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:55.131410 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c323637-6401-4c32-b94e-dc42fe7873dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:55.132079 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c323637-6401-4c32-b94e-dc42fe7873dd None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:55.132362 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c323637-6401-4c32-b94e-dc42fe7873dd None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:55.132627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c323637-6401-4c32-b94e-dc42fe7873dd None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:55.133363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c323637-6401-4c32-b94e-dc42fe7873dd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:55.134873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c323637-6401-4c32-b94e-dc42fe7873dd None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:55.161157 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 30/56] 10.222.0.44 () {60 vars in 1140 bytes} [Thu Jul 21 05:29:54 2022] POST /identity/v3/projects => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:29:55.238160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56fc4c-6290-438f-a144-f02429832216 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:55.257675 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:55.277189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355af522-5f1f-4a5a-8edb-0dda12436792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:55.277884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355af522-5f1f-4a5a-8edb-0dda12436792 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:55.278178 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355af522-5f1f-4a5a-8edb-0dda12436792 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:55.278449 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355af522-5f1f-4a5a-8edb-0dda12436792 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:55.279360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355af522-5f1f-4a5a-8edb-0dda12436792 None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.domain.id=default, user.id=3b3cc4a53587419894e0f708d1e60e89) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:29:55.279680 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355af522-5f1f-4a5a-8edb-0dda12436792 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:55.281222 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355af522-5f1f-4a5a-8edb-0dda12436792 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:55.288661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 27/57] 10.222.0.44 () {56 vars in 1322 bytes} [Thu Jul 21 05:29:55 2022] GET /identity/v3/role_assignments?user.id=3b3cc4a53587419894e0f708d1e60e89&scope.domain.id=default&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:29:57.185967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856c078-f0e2-40da-b36b-a1c867abeb2c None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:57.186519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856c078-f0e2-40da-b36b-a1c867abeb2c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:57.186519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856c078-f0e2-40da-b36b-a1c867abeb2c None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:57.187944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 31/58] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:29:57.196738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f7776-d1cc-4945-bbfd-9c9914f466f9 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:57.197443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f7776-d1cc-4945-bbfd-9c9914f466f9 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:57.197611 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f7776-d1cc-4945-bbfd-9c9914f466f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:57.220748 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-881f7776-d1cc-4945-bbfd-9c9914f466f9 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:29:57.221936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-881f7776-d1cc-4945-bbfd-9c9914f466f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:29:57.294867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 28/59] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:29:57.334286 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856c078-f0e2-40da-b36b-a1c867abeb2c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:57.457744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856c078-f0e2-40da-b36b-a1c867abeb2c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:57.492532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbeb7f-e578-47c2-a5b7-e17644ad25a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:57.493318 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbeb7f-e578-47c2-a5b7-e17644ad25a6 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:57.493632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbeb7f-e578-47c2-a5b7-e17644ad25a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:57.493910 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbeb7f-e578-47c2-a5b7-e17644ad25a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:57.495212 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddbeb7f-e578-47c2-a5b7-e17644ad25a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:57.496685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddbeb7f-e578-47c2-a5b7-e17644ad25a6 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:57.498327 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 32/60] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:29:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:29:57.504334 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f7776-d1cc-4945-bbfd-9c9914f466f9 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:29:57.604937 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881f7776-d1cc-4945-bbfd-9c9914f466f9 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:29:57.615346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:29:57.643731 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199925c-859c-4897-a54a-42761e8e7cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:29:57.644417 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199925c-859c-4897-a54a-42761e8e7cb4 None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:29:57.644705 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199925c-859c-4897-a54a-42761e8e7cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:29:57.644959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199925c-859c-4897-a54a-42761e8e7cb4 None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:29:57.645672 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199925c-859c-4897-a54a-42761e8e7cb4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:29:57.647171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199925c-859c-4897-a54a-42761e8e7cb4 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:29:57.674127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 29/61] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:29:57 2022] POST /identity/v3/users => generated 275 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:30:00.793333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb79d0-9556-44ea-9a14-e1c6097a6754 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:00.793888 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb79d0-9556-44ea-9a14-e1c6097a6754 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:00.793888 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb79d0-9556-44ea-9a14-e1c6097a6754 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:00.795676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 33/62] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:00.800970 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44318c40-f218-4e62-8920-2e670439ca54 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:00.801374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44318c40-f218-4e62-8920-2e670439ca54 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:00.801563 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44318c40-f218-4e62-8920-2e670439ca54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:00.828080 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-44318c40-f218-4e62-8920-2e670439ca54 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:00.829298 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-44318c40-f218-4e62-8920-2e670439ca54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:00.922668 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 30/63] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:00.957914 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb79d0-9556-44ea-9a14-e1c6097a6754 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:00.976874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:00.984233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9e949-3055-475f-a134-71fb550adc6b None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:00.984526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9e949-3055-475f-a134-71fb550adc6b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:00.984807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9e949-3055-475f-a134-71fb550adc6b None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:00.986230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 31/64] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:00.992037 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b3ccb-8c42-4ff5-bc87-a72b13e2f04d None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:00.992331 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b3ccb-8c42-4ff5-bc87-a72b13e2f04d None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:00.992675 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b3ccb-8c42-4ff5-bc87-a72b13e2f04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:01.014855 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-9a3b3ccb-8c42-4ff5-bc87-a72b13e2f04d None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:01.016100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9a3b3ccb-8c42-4ff5-bc87-a72b13e2f04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:01.090175 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb79d0-9556-44ea-9a14-e1c6097a6754 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:01.103637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 32/65] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:01.105817 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce40ae-3903-4f02-8c3a-fefbd516dd31 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:01.106211 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce40ae-3903-4f02-8c3a-fefbd516dd31 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:01.106844 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce40ae-3903-4f02-8c3a-fefbd516dd31 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:01.108517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 33/66] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:01.113723 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82063998-3562-4880-9503-650b51ce8b0c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:01.114228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82063998-3562-4880-9503-650b51ce8b0c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:01.114681 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82063998-3562-4880-9503-650b51ce8b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:01.133164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-82063998-3562-4880-9503-650b51ce8b0c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:01.133410 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84129351-8a27-449a-954b-8a2e167c732e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:01.134525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84129351-8a27-449a-954b-8a2e167c732e None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:01.134917 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-82063998-3562-4880-9503-650b51ce8b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:01.135117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84129351-8a27-449a-954b-8a2e167c732e None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:01.135529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84129351-8a27-449a-954b-8a2e167c732e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:01.146564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84129351-8a27-449a-954b-8a2e167c732e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:01.154506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-84129351-8a27-449a-954b-8a2e167c732e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:01.154506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:01.154506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:01.154506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:01.154506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:01.154506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:01.154506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:01.154506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:01.154506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:01.154506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:01.154506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:01.154506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:01.154506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:01.154506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:01.154506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:01.154506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:01.155553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:01.155553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:01.155553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:01.155553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:01.155553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:01.155553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:01.155553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:01.155553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:01.155553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:01.155553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:01.155553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:01.155553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:01.155553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:01.155553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:01.155553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:01.155553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:01.156519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:01.156519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:01.156519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:01.156519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:01.156519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:01.156519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:01.156519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:01.156519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:01.156519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:01.156519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:01.156519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:01.156519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:01.158785 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 34/67] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:30:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:01.159619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:01.160009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84129351-8a27-449a-954b-8a2e167c732e None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:01.219661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 34/68] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:01.220233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82063998-3562-4880-9503-650b51ce8b0c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:01.252358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:01.292173 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84129351-8a27-449a-954b-8a2e167c732e None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:01.336910 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926b737-9336-4de1-8537-b540f4b126ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:01.337766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926b737-9336-4de1-8537-b540f4b126ea None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:01.338055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926b737-9336-4de1-8537-b540f4b126ea None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:01.338335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926b737-9336-4de1-8537-b540f4b126ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:01.348326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82063998-3562-4880-9503-650b51ce8b0c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:01.348820 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926b737-9336-4de1-8537-b540f4b126ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:01.356407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-5926b737-9336-4de1-8537-b540f4b126ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:01.356407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:01.356407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:01.356407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:01.356407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:01.356407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:01.356407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:01.356407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:01.356407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:01.356407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:01.356407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:01.356407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:01.356407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:01.356407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:01.356407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:01.356407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:01.357445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:01.357445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:01.357445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:01.357445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:01.357445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:01.357445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:01.357445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:01.357445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:01.357445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:01.357445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:01.357445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:01.357445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:01.357445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:01.357445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:01.357445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:01.357445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:01.363134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:01.363134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:01.363134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:01.363134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:01.363134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:01.363134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:01.363134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:01.363134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:01.363134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:01.363134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:01.363134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:01.363134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:01.363134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 35/69] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:01.363134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926b737-9336-4de1-8537-b540f4b126ea None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:01.392870 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5934f4ba-ca89-48b1-b1c7-4edd6d645892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:01.393761 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934f4ba-ca89-48b1-b1c7-4edd6d645892 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:01.394139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934f4ba-ca89-48b1-b1c7-4edd6d645892 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:01.394495 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5934f4ba-ca89-48b1-b1c7-4edd6d645892 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:01.395372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934f4ba-ca89-48b1-b1c7-4edd6d645892 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:01.396279 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934f4ba-ca89-48b1-b1c7-4edd6d645892 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:01.397122 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5934f4ba-ca89-48b1-b1c7-4edd6d645892 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:01.406460 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 35/70] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:01.407701 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9125ae-ac2f-44b2-8a3f-1a48de46613c None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:01.407998 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9125ae-ac2f-44b2-8a3f-1a48de46613c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:01.408280 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9125ae-ac2f-44b2-8a3f-1a48de46613c None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:01.409895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 36/71] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:01.410610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9125ae-ac2f-44b2-8a3f-1a48de46613c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:01.430865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:01.466311 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926b737-9336-4de1-8537-b540f4b126ea None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:01.494149 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:01.499333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0ac2a-3d59-4300-b817-01388dd62071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:01.500144 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0ac2a-3d59-4300-b817-01388dd62071 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:01.500424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0ac2a-3d59-4300-b817-01388dd62071 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:01.500699 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0ac2a-3d59-4300-b817-01388dd62071 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:01.510637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0ac2a-3d59-4300-b817-01388dd62071 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:01.512066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9125ae-ac2f-44b2-8a3f-1a48de46613c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:01.518519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-fde0ac2a-3d59-4300-b817-01388dd62071 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:01.518519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:01.518519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:01.518519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:01.518519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:01.518519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:01.518519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:01.518519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:01.518519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:01.518519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:01.518519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:01.518519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:01.518519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:01.518519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:01.518519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:01.518519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:01.519611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:01.519611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:01.519611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:01.519611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:01.519611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:01.519611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:01.519611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:01.519611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:01.519611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:01.519611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:01.519611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:01.519611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:01.519611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:01.519611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:01.519611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:01.519611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:01.520546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:01.520546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:01.520546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:01.520546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:01.520546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:01.520546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:01.520546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:01.520546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:01.520546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:01.520546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:01.520546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:01.520546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:01.520546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 36/72] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:30:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:01.520546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0ac2a-3d59-4300-b817-01388dd62071 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:01.548193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2508c6-a726-47a5-bd5b-f71dfa33c977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:01.548866 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2508c6-a726-47a5-bd5b-f71dfa33c977 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:01.549156 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2508c6-a726-47a5-bd5b-f71dfa33c977 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:01.549438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2508c6-a726-47a5-bd5b-f71dfa33c977 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:01.550195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2508c6-a726-47a5-bd5b-f71dfa33c977 None admin] RBAC: Adding query filter params (name=member) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:01.550509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2508c6-a726-47a5-bd5b-f71dfa33c977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:01.551943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2508c6-a726-47a5-bd5b-f71dfa33c977 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:01.560705 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 37/73] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:01.561944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dfb47-71ba-47fd-9913-3728276fcb3d None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:01.562238 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dfb47-71ba-47fd-9913-3728276fcb3d None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:01.562519 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dfb47-71ba-47fd-9913-3728276fcb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:01.578424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-ee2dfb47-71ba-47fd-9913-3728276fcb3d None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:01.579652 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ee2dfb47-71ba-47fd-9913-3728276fcb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:01.618912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0ac2a-3d59-4300-b817-01388dd62071 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:01.643683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 38/74] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:01.644992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9c89d-d00c-4f69-a46f-e33e0a5dcc4c None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:01.645288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9c89d-d00c-4f69-a46f-e33e0a5dcc4c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:01.645573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9c89d-d00c-4f69-a46f-e33e0a5dcc4c None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:01.646995 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 39/75] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:01.647592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9c89d-d00c-4f69-a46f-e33e0a5dcc4c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:01.659681 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:01.663123 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab58209-2134-4ef2-83b2-896e7e8df562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:01.664880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab58209-2134-4ef2-83b2-896e7e8df562 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:01.665585 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab58209-2134-4ef2-83b2-896e7e8df562 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:01.666101 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab58209-2134-4ef2-83b2-896e7e8df562 None admin] PATH_INFO: `/v3/users/3b3cc4a53587419894e0f708d1e60e89` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:01.669822 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab58209-2134-4ef2-83b2-896e7e8df562 None admin] RBAC: Authorizing `identity:get_user(user_id=3b3cc4a53587419894e0f708d1e60e89)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:01.674645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab58209-2134-4ef2-83b2-896e7e8df562 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:01.678571 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 37/76] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:01 2022] GET /identity/v3/users/3b3cc4a53587419894e0f708d1e60e89 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:01.681702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab58209-2134-4ef2-83b2-896e7e8df562 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:01.743566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:01.792753 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab58209-2134-4ef2-83b2-896e7e8df562 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:01.794915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9c89d-d00c-4f69-a46f-e33e0a5dcc4c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:01.829427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbab9c-78ef-4bf0-af44-6bc70643c3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:01.830316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbab9c-78ef-4bf0-af44-6bc70643c3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:01.830602 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbab9c-78ef-4bf0-af44-6bc70643c3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:01.830876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbab9c-78ef-4bf0-af44-6bc70643c3f4 None admin] PATH_INFO: `/v3/users/c23f5930d01c4d6f819cc9308da0b14e` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:01.831174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ba52f-9114-4215-b8cd-9e5dd3bd12bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:01.831853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ba52f-9114-4215-b8cd-9e5dd3bd12bb None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:01.832140 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ba52f-9114-4215-b8cd-9e5dd3bd12bb None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:01.832415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ba52f-9114-4215-b8cd-9e5dd3bd12bb None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:01.833163 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ba52f-9114-4215-b8cd-9e5dd3bd12bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:01.833467 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ba52f-9114-4215-b8cd-9e5dd3bd12bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:01.834865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ba52f-9114-4215-b8cd-9e5dd3bd12bb None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:01.847426 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 40/77] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:30:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:01.847426 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c548260-efe2-4a1a-8223-bcf897085051 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:01.847426 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c548260-efe2-4a1a-8223-bcf897085051 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:01.847426 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c548260-efe2-4a1a-8223-bcf897085051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:01.849908 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbab9c-78ef-4bf0-af44-6bc70643c3f4 None admin] RBAC: Authorizing `identity:get_user(user_id=c23f5930d01c4d6f819cc9308da0b14e)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:01.851429 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbab9c-78ef-4bf0-af44-6bc70643c3f4 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:01.853335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 38/78] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:01 2022] GET /identity/v3/users/c23f5930d01c4d6f819cc9308da0b14e => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:01.854113 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbab9c-78ef-4bf0-af44-6bc70643c3f4 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:01.863811 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-7c548260-efe2-4a1a-8223-bcf897085051 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:01.865022 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7c548260-efe2-4a1a-8223-bcf897085051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:01.939498 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 41/79] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:01.943940 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c548260-efe2-4a1a-8223-bcf897085051 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:01.946670 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:01.972338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbab9c-78ef-4bf0-af44-6bc70643c3f4 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:01.994954 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:02.015081 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb873cc-0c40-430c-941e-671faf0527d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:02.015912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb873cc-0c40-430c-941e-671faf0527d6 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:02.016206 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb873cc-0c40-430c-941e-671faf0527d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:02.016484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb873cc-0c40-430c-941e-671faf0527d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:02.026707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb873cc-0c40-430c-941e-671faf0527d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:02.034371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-3fb873cc-0c40-430c-941e-671faf0527d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:02.034371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:02.034371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:02.034371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:02.034371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:02.034371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:02.034371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:02.034371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:02.034371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:02.034371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:02.034371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:02.034371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:02.034371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:02.034371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:02.034371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:02.034371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:02.035349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:02.035349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:02.035349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:02.035349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:02.035349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:02.035349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:02.035349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:02.035349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:02.035349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:02.035349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:02.035349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:02.035349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:02.035349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:02.035349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:02.035349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:02.035349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:02.036114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:02.036114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:02.036114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:02.036114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:02.036114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:02.036114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:02.036114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:02.036114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:02.036114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:02.036114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:02.036114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:02.036114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:02.036114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 39/80] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:02.036830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb873cc-0c40-430c-941e-671faf0527d6 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:02.053000 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c548260-efe2-4a1a-8223-bcf897085051 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:02.085442 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057221f2-5565-4e03-8bb9-ea71adac6cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:02.086097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057221f2-5565-4e03-8bb9-ea71adac6cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:02.086376 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057221f2-5565-4e03-8bb9-ea71adac6cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:02.086648 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057221f2-5565-4e03-8bb9-ea71adac6cf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:02.133666 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb873cc-0c40-430c-941e-671faf0527d6 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:02.150646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057221f2-5565-4e03-8bb9-ea71adac6cf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:02.152167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057221f2-5565-4e03-8bb9-ea71adac6cf2 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:02.155458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:02.167625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b260172-2c77-4278-9eaf-2c1fa1c70992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:02.168776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b260172-2c77-4278-9eaf-2c1fa1c70992 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:02.169197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b260172-2c77-4278-9eaf-2c1fa1c70992 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:02.169654 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b260172-2c77-4278-9eaf-2c1fa1c70992 None admin] PATH_INFO: `/v3/users/c23f5930d01c4d6f819cc9308da0b14e` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:02.171575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b260172-2c77-4278-9eaf-2c1fa1c70992 None admin] RBAC: Authorizing `identity:get_user(user_id=c23f5930d01c4d6f819cc9308da0b14e)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:02.173067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b260172-2c77-4278-9eaf-2c1fa1c70992 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:02.174981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 40/81] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:02 2022] GET /identity/v3/users/c23f5930d01c4d6f819cc9308da0b14e => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:02.179944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b260172-2c77-4278-9eaf-2c1fa1c70992 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:02.225505 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:02.251288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 42/82] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:02.252136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057221f2-5565-4e03-8bb9-ea71adac6cf2 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:02.280837 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b260172-2c77-4278-9eaf-2c1fa1c70992 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:02.313915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d6654-f6f6-4880-959b-64a15538908a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:02.314505 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d6654-f6f6-4880-959b-64a15538908a None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:02.314790 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d6654-f6f6-4880-959b-64a15538908a None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:02.315090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d6654-f6f6-4880-959b-64a15538908a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:02.346455 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057221f2-5565-4e03-8bb9-ea71adac6cf2 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:02.347974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:02.379789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d6654-f6f6-4880-959b-64a15538908a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:02.381180 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875cd2d-165a-4cc0-abb9-d9a7d06cd5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:02.381362 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d6654-f6f6-4880-959b-64a15538908a None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:02.381969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875cd2d-165a-4cc0-abb9-d9a7d06cd5fb None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:02.382266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875cd2d-165a-4cc0-abb9-d9a7d06cd5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:02.382533 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875cd2d-165a-4cc0-abb9-d9a7d06cd5fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:02.383731 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875cd2d-165a-4cc0-abb9-d9a7d06cd5fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:02.385217 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875cd2d-165a-4cc0-abb9-d9a7d06cd5fb None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:02.387184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 43/83] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:30:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:02.387646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875cd2d-165a-4cc0-abb9-d9a7d06cd5fb None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:02.430714 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:02.485376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 41/84] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:02.486248 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d6654-f6f6-4880-959b-64a15538908a None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:02.506493 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875cd2d-165a-4cc0-abb9-d9a7d06cd5fb None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:02.541207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552667b6-cea9-4450-9e4e-47a4d4bfb6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:02.541888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552667b6-cea9-4450-9e4e-47a4d4bfb6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:02.542171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552667b6-cea9-4450-9e4e-47a4d4bfb6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:02.542477 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552667b6-cea9-4450-9e4e-47a4d4bfb6a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:02.543243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552667b6-cea9-4450-9e4e-47a4d4bfb6a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:02.543555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552667b6-cea9-4450-9e4e-47a4d4bfb6a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:02.544978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552667b6-cea9-4450-9e4e-47a4d4bfb6a4 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:02.553376 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 44/85] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:02.554091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552667b6-cea9-4450-9e4e-47a4d4bfb6a4 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:02.586399 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:02.591192 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d6654-f6f6-4880-959b-64a15538908a None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:02.624098 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db1164-6c79-4cf8-a026-78c02daf0455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:02.624764 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db1164-6c79-4cf8-a026-78c02daf0455 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:02.625051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db1164-6c79-4cf8-a026-78c02daf0455 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:02.625331 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db1164-6c79-4cf8-a026-78c02daf0455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:02.640194 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:02.653901 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552667b6-cea9-4450-9e4e-47a4d4bfb6a4 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:02.687257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6f150-dbe5-47ba-bd3b-ae5b7c7a7aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:02.688140 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6f150-dbe5-47ba-bd3b-ae5b7c7a7aaf None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:02.688430 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6f150-dbe5-47ba-bd3b-ae5b7c7a7aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:02.688710 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6f150-dbe5-47ba-bd3b-ae5b7c7a7aaf None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:02.689903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6f150-dbe5-47ba-bd3b-ae5b7c7a7aaf None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:02.691431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c6f150-dbe5-47ba-bd3b-ae5b7c7a7aaf None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:02.692274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db1164-6c79-4cf8-a026-78c02daf0455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:02.693150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 45/86] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:02 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:02.693748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db1164-6c79-4cf8-a026-78c02daf0455 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:02.693898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6f150-dbe5-47ba-bd3b-ae5b7c7a7aaf None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:02.789642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:02.792997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6f150-dbe5-47ba-bd3b-ae5b7c7a7aaf None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:02.797053 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 42/87] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:02.797901 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db1164-6c79-4cf8-a026-78c02daf0455 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:02.832876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd1e78-e675-4ed5-af21-5c0d7ece3a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:02.833527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd1e78-e675-4ed5-af21-5c0d7ece3a58 None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:02.833833 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd1e78-e675-4ed5-af21-5c0d7ece3a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:02.834113 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd1e78-e675-4ed5-af21-5c0d7ece3a58 None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:02.834843 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd1e78-e675-4ed5-af21-5c0d7ece3a58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:02.836484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd1e78-e675-4ed5-af21-5c0d7ece3a58 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:02.848183 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:02.856671 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 46/88] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:30:02 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:30:02.857498 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd1e78-e675-4ed5-af21-5c0d7ece3a58 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:02.928037 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db1164-6c79-4cf8-a026-78c02daf0455 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:02.966386 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c43362-d9bb-43b9-ab12-5a6df0d79df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:02.967403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c43362-d9bb-43b9-ab12-5a6df0d79df4 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:02.967760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c43362-d9bb-43b9-ab12-5a6df0d79df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:02.968036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c43362-d9bb-43b9-ab12-5a6df0d79df4 None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:02.969287 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c43362-d9bb-43b9-ab12-5a6df0d79df4 None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:02.969391 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd1e78-e675-4ed5-af21-5c0d7ece3a58 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:02.970807 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c43362-d9bb-43b9-ab12-5a6df0d79df4 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:02.972861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 43/89] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:02 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:02.973673 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c43362-d9bb-43b9-ab12-5a6df0d79df4 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:03.017066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5be40-4e3d-4b79-b8df-06dcc708cfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:03.017998 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5be40-4e3d-4b79-b8df-06dcc708cfdb None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:03.018286 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5be40-4e3d-4b79-b8df-06dcc708cfdb None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:03.018553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5be40-4e3d-4b79-b8df-06dcc708cfdb None admin] PATH_INFO: `/v3/users/c23f5930d01c4d6f819cc9308da0b14e` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:03.019796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5be40-4e3d-4b79-b8df-06dcc708cfdb None admin] RBAC: Authorizing `identity:get_user(user_id=c23f5930d01c4d6f819cc9308da0b14e)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:03.021292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5be40-4e3d-4b79-b8df-06dcc708cfdb None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:03.028417 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 47/90] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:02 2022] GET /identity/v3/users/c23f5930d01c4d6f819cc9308da0b14e => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:03.028417 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5be40-4e3d-4b79-b8df-06dcc708cfdb None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:03.052310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:03.090892 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c43362-d9bb-43b9-ab12-5a6df0d79df4 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:03.109181 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:03.143705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d29fa-512e-49c5-a180-fb26ddd75a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:03.144539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d29fa-512e-49c5-a180-fb26ddd75a5d None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:03.144846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d29fa-512e-49c5-a180-fb26ddd75a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:03.145107 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d29fa-512e-49c5-a180-fb26ddd75a5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:03.145960 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d29fa-512e-49c5-a180-fb26ddd75a5d None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=db74f3f0803a45359025fb277815314b, user.id=3b3cc4a53587419894e0f708d1e60e89) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:03.146071 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5be40-4e3d-4b79-b8df-06dcc708cfdb None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:03.146280 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d29fa-512e-49c5-a180-fb26ddd75a5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:03.148005 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d29fa-512e-49c5-a180-fb26ddd75a5d None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:03.158344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 44/91] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:02 2022] GET /identity/v3/role_assignments?user.id=3b3cc4a53587419894e0f708d1e60e89&scope.project.id=db74f3f0803a45359025fb277815314b&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:03.159642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d29fa-512e-49c5-a180-fb26ddd75a5d None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:03.195946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97068802-6a9b-431f-8373-91e823ebc042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:03.197265 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97068802-6a9b-431f-8373-91e823ebc042 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:03.197559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97068802-6a9b-431f-8373-91e823ebc042 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:03.197879 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97068802-6a9b-431f-8373-91e823ebc042 None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:03.200204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97068802-6a9b-431f-8373-91e823ebc042 None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:03.201721 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97068802-6a9b-431f-8373-91e823ebc042 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:03.205032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 48/92] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:03 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:03.205974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97068802-6a9b-431f-8373-91e823ebc042 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:03.299456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d29fa-512e-49c5-a180-fb26ddd75a5d None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:03.358491 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:03.367938 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40122a0c-f5ad-4f7c-9ead-65557d173fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:03.368721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40122a0c-f5ad-4f7c-9ead-65557d173fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:03.369011 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40122a0c-f5ad-4f7c-9ead-65557d173fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:03.369267 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40122a0c-f5ad-4f7c-9ead-65557d173fd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:03.370118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40122a0c-f5ad-4f7c-9ead-65557d173fd8 None admin] RBAC: Adding query filter params (role.id=b59c5b350c854281838a22884fd8f32f, scope.project.id=db74f3f0803a45359025fb277815314b, user.id=c23f5930d01c4d6f819cc9308da0b14e) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:03.370427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40122a0c-f5ad-4f7c-9ead-65557d173fd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:03.371925 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40122a0c-f5ad-4f7c-9ead-65557d173fd8 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:03.377471 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97068802-6a9b-431f-8373-91e823ebc042 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:03.389029 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 45/93] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:03 2022] GET /identity/v3/role_assignments?user.id=c23f5930d01c4d6f819cc9308da0b14e&scope.project.id=db74f3f0803a45359025fb277815314b&role.id=b59c5b350c854281838a22884fd8f32f => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:03.389884 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40122a0c-f5ad-4f7c-9ead-65557d173fd8 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:03.443923 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d3403-4665-406c-92d0-17a9e02dd5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:03.444618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d3403-4665-406c-92d0-17a9e02dd5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:03.444910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d3403-4665-406c-92d0-17a9e02dd5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:03.445195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d3403-4665-406c-92d0-17a9e02dd5e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:03.491375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:03.576852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d3403-4665-406c-92d0-17a9e02dd5e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:03.578368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d3403-4665-406c-92d0-17a9e02dd5e3 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:03.637223 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40122a0c-f5ad-4f7c-9ead-65557d173fd8 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:03.691807 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91ed21-a013-4672-a598-3ab9e047b752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:03.692493 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91ed21-a013-4672-a598-3ab9e047b752 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:03.692783 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91ed21-a013-4672-a598-3ab9e047b752 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:03.693068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91ed21-a013-4672-a598-3ab9e047b752 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:03.693924 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91ed21-a013-4672-a598-3ab9e047b752 None admin] RBAC: Adding query filter params (role.id=a43ba93f358b4b1a8a445851a5a216a0, scope.project.id=db74f3f0803a45359025fb277815314b, user.id=c23f5930d01c4d6f819cc9308da0b14e) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:03.694242 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91ed21-a013-4672-a598-3ab9e047b752 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:03.695754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91ed21-a013-4672-a598-3ab9e047b752 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:03.707811 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 46/94] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:03 2022] GET /identity/v3/role_assignments?user.id=c23f5930d01c4d6f819cc9308da0b14e&scope.project.id=db74f3f0803a45359025fb277815314b&role.id=a43ba93f358b4b1a8a445851a5a216a0 => generated 259 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:03.760012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:03.776047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 49/95] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:03.782240 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91ed21-a013-4672-a598-3ab9e047b752 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:03.897834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91ed21-a013-4672-a598-3ab9e047b752 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:03.905792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:03.938447 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee326c4-e19f-4702-a7fd-058af05fb74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:03.939364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee326c4-e19f-4702-a7fd-058af05fb74c None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:03.939655 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee326c4-e19f-4702-a7fd-058af05fb74c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:03.939935 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee326c4-e19f-4702-a7fd-058af05fb74c None admin] PATH_INFO: `/v3/projects/d8f593f0a0934e5f9f2baf99016b98a7` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:03.941153 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee326c4-e19f-4702-a7fd-058af05fb74c None admin] RBAC: Authorizing `identity:get_project(project_id=d8f593f0a0934e5f9f2baf99016b98a7)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:03.942692 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee326c4-e19f-4702-a7fd-058af05fb74c None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:03.944632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 47/96] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:03 2022] GET /identity/v3/projects/d8f593f0a0934e5f9f2baf99016b98a7 => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:03.950814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d3403-4665-406c-92d0-17a9e02dd5e3 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:04.077743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d3403-4665-406c-92d0-17a9e02dd5e3 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:04.123736 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d0d99-2b8b-4935-b45c-f9ff8b12cf33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:04.124421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d0d99-2b8b-4935-b45c-f9ff8b12cf33 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:04.124713 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d0d99-2b8b-4935-b45c-f9ff8b12cf33 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:04.125003 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d0d99-2b8b-4935-b45c-f9ff8b12cf33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:04.125848 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d0d99-2b8b-4935-b45c-f9ff8b12cf33 None admin] RBAC: Adding query filter params (role.id=b59c5b350c854281838a22884fd8f32f, scope.project.id=d8f593f0a0934e5f9f2baf99016b98a7, user.id=c23f5930d01c4d6f819cc9308da0b14e) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:04.126168 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d0d99-2b8b-4935-b45c-f9ff8b12cf33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:04.127700 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d0d99-2b8b-4935-b45c-f9ff8b12cf33 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:04.135421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 50/97] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:03 2022] GET /identity/v3/role_assignments?user.id=c23f5930d01c4d6f819cc9308da0b14e&scope.project.id=d8f593f0a0934e5f9f2baf99016b98a7&role.id=b59c5b350c854281838a22884fd8f32f => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:05.512569 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78036e62-87f3-4526-a44d-ce90a88b3279 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:05.513096 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78036e62-87f3-4526-a44d-ce90a88b3279 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:05.513171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78036e62-87f3-4526-a44d-ce90a88b3279 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:05.514699 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 48/98] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:05.520013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12910feb-d394-422b-a22d-6e7f98ae6769 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:05.520408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12910feb-d394-422b-a22d-6e7f98ae6769 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:05.520595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12910feb-d394-422b-a22d-6e7f98ae6769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:05.548043 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-12910feb-d394-422b-a22d-6e7f98ae6769 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:05.549266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-12910feb-d394-422b-a22d-6e7f98ae6769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:05.614104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:05.638482 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 51/99] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:05.673534 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78036e62-87f3-4526-a44d-ce90a88b3279 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:05.808273 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78036e62-87f3-4526-a44d-ce90a88b3279 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:05.858488 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f6830-680e-4ed1-9426-a7179dd80c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:05.859352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f6830-680e-4ed1-9426-a7179dd80c5c None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:05.859646 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f6830-680e-4ed1-9426-a7179dd80c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:05.859933 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f6830-680e-4ed1-9426-a7179dd80c5c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:05.870860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f6830-680e-4ed1-9426-a7179dd80c5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:05.878460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-455f6830-680e-4ed1-9426-a7179dd80c5c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:05.878460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:05.878460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:05.878460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:05.878460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:05.878460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:05.878460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:05.878460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:05.878460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:05.878460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:05.878460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:05.878460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:05.878460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:05.878460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:05.878460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:05.878460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:05.879375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:05.879375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:05.879375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:05.879375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:05.879375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:05.879375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:05.879375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:05.879375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:05.879375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:05.879375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:05.879375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:05.879375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:05.879375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:05.879375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:05.879375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:05.879375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:05.880130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:05.880130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:05.880130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:05.880130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:05.880130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:05.880130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:05.880130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:05.880130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:05.880130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:05.880130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:05.880130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:05.880130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:05.880130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 49/100] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:05.885713 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12910feb-d394-422b-a22d-6e7f98ae6769 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:06.014916 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12910feb-d394-422b-a22d-6e7f98ae6769 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:06.062603 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0ca25-9dcd-40e6-9594-7fac4752a5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:06.063671 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0ca25-9dcd-40e6-9594-7fac4752a5cc None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:06.064181 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0ca25-9dcd-40e6-9594-7fac4752a5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:06.064653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0ca25-9dcd-40e6-9594-7fac4752a5cc None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:06.065608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0ca25-9dcd-40e6-9594-7fac4752a5cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:06.066113 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0ca25-9dcd-40e6-9594-7fac4752a5cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:06.067769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0ca25-9dcd-40e6-9594-7fac4752a5cc None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:06.077399 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 52/101] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:06.082662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f6830-680e-4ed1-9426-a7179dd80c5c None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:06.122751 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:06.181462 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dfea2-6bb5-456e-9aee-5b00a4feaee2 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:06.181809 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dfea2-6bb5-456e-9aee-5b00a4feaee2 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:06.182099 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dfea2-6bb5-456e-9aee-5b00a4feaee2 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:06.183548 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 53/102] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:30:06.213767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f6830-680e-4ed1-9426-a7179dd80c5c None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:06.225580 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276e480-b55d-4618-9938-d3f595ba37dc None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:06.225881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276e480-b55d-4618-9938-d3f595ba37dc None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:06.226183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276e480-b55d-4618-9938-d3f595ba37dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:06.255667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fba261-d67b-4b5f-80f7-8237839ca8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:06.256916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fba261-d67b-4b5f-80f7-8237839ca8ce None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:06.257420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fba261-d67b-4b5f-80f7-8237839ca8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:06.257894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fba261-d67b-4b5f-80f7-8237839ca8ce None admin] PATH_INFO: `/v3/users/aa86fe483ab441bcbb42b09b4c81accc` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:06.258181 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-9276e480-b55d-4618-9938-d3f595ba37dc None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:06.275040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9276e480-b55d-4618-9938-d3f595ba37dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:06.284271 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fba261-d67b-4b5f-80f7-8237839ca8ce None admin] RBAC: Authorizing `identity:get_user(user_id=aa86fe483ab441bcbb42b09b4c81accc)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:06.286179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fba261-d67b-4b5f-80f7-8237839ca8ce None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:06.288609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 50/103] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:06 2022] GET /identity/v3/users/aa86fe483ab441bcbb42b09b4c81accc => generated 289 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:06.294255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fba261-d67b-4b5f-80f7-8237839ca8ce None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:06.329937 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:06.352850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 54/104] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:06.354606 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bfd1b-8749-477b-ae12-ff6673de7e26 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:06.355244 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bfd1b-8749-477b-ae12-ff6673de7e26 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:06.355943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bfd1b-8749-477b-ae12-ff6673de7e26 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:06.357583 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 55/105] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:06.367065 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fa866-eb6c-47d9-861c-e39e4de534fe None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:06.367065 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fa866-eb6c-47d9-861c-e39e4de534fe None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:06.367065 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fa866-eb6c-47d9-861c-e39e4de534fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:06.387282 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-427fa866-eb6c-47d9-861c-e39e4de534fe None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:06.388818 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-427fa866-eb6c-47d9-861c-e39e4de534fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:06.448150 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:06.461645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fba261-d67b-4b5f-80f7-8237839ca8ce None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:06.466726 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 56/106] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:06.473566 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427fa866-eb6c-47d9-861c-e39e4de534fe None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:06.514931 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fbab2-698b-4f63-98ba-1ddeb26c3763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:06.515636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fbab2-698b-4f63-98ba-1ddeb26c3763 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:06.515981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fbab2-698b-4f63-98ba-1ddeb26c3763 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:06.516264 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fbab2-698b-4f63-98ba-1ddeb26c3763 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:06.597110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427fa866-eb6c-47d9-861c-e39e4de534fe None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:06.602867 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fbab2-698b-4f63-98ba-1ddeb26c3763 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:06.604441 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7fbab2-698b-4f63-98ba-1ddeb26c3763 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:06.608102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:06.633573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf15cf-b5a5-4864-b1f7-3ee59bf5b265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:06.635346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf15cf-b5a5-4864-b1f7-3ee59bf5b265 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:06.636044 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf15cf-b5a5-4864-b1f7-3ee59bf5b265 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:06.636720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf15cf-b5a5-4864-b1f7-3ee59bf5b265 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:06.647647 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf15cf-b5a5-4864-b1f7-3ee59bf5b265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:06.655666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-4ddf15cf-b5a5-4864-b1f7-3ee59bf5b265 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:06.655666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:06.655666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:06.655666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:06.655666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:06.655666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:06.655666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:06.655666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:06.655666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:06.655666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:06.655666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:06.655666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:06.655666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:06.655666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:06.655666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:06.655666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:06.656560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:06.656560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:06.656560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:06.656560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:06.656560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:06.656560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:06.656560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:06.656560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:06.656560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:06.656560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:06.656560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:06.656560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:06.656560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:06.656560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:06.656560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:06.656560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:06.657429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:06.657429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:06.657429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:06.657429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:06.657429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:06.657429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:06.657429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:06.657429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:06.657429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:06.657429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:06.657429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:06.657429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:06.659819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 57/107] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:30:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:06.661387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caedf4f-2ed8-48ca-a5a6-5707a032fd16 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:06.661876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caedf4f-2ed8-48ca-a5a6-5707a032fd16 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:06.662464 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caedf4f-2ed8-48ca-a5a6-5707a032fd16 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:06.669140 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 58/108] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:06.670402 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caedf4f-2ed8-48ca-a5a6-5707a032fd16 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:06.702336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:06.714428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 51/109] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 05:30:06.715806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbb0f1-bc29-4ba6-a059-c6f10cbfe96a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:06.716102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbb0f1-bc29-4ba6-a059-c6f10cbfe96a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:06.716381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbb0f1-bc29-4ba6-a059-c6f10cbfe96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:06.734247 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-86dbb0f1-bc29-4ba6-a059-c6f10cbfe96a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:06.735468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-86dbb0f1-bc29-4ba6-a059-c6f10cbfe96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:06.791720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caedf4f-2ed8-48ca-a5a6-5707a032fd16 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:06.831037 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 52/110] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:06.831257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cb5f9-2f79-42a5-b6fa-c24678f06041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:06.831257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cb5f9-2f79-42a5-b6fa-c24678f06041 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:06.831257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cb5f9-2f79-42a5-b6fa-c24678f06041 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:06.831257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cb5f9-2f79-42a5-b6fa-c24678f06041 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:06.831527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cb5f9-2f79-42a5-b6fa-c24678f06041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:06.831937 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cb5f9-2f79-42a5-b6fa-c24678f06041 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:06.833077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbb0f1-bc29-4ba6-a059-c6f10cbfe96a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:06.833698 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 59/111] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:30:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:06.837086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cb5f9-2f79-42a5-b6fa-c24678f06041 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:06.889130 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:06.956606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbb0f1-bc29-4ba6-a059-c6f10cbfe96a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:06.958873 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cb5f9-2f79-42a5-b6fa-c24678f06041 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:06.980606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:06.993138 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fdd3e-7563-4db1-9011-2d313d9fb2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:06.993495 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93686a-b87a-4389-9d47-7b25810aeb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:06.993888 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fdd3e-7563-4db1-9011-2d313d9fb2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:06.994175 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fdd3e-7563-4db1-9011-2d313d9fb2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:06.994379 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93686a-b87a-4389-9d47-7b25810aeb9a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:06.994473 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fdd3e-7563-4db1-9011-2d313d9fb2d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:06.994662 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93686a-b87a-4389-9d47-7b25810aeb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:06.994940 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b93686a-b87a-4389-9d47-7b25810aeb9a None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:06.995230 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405fdd3e-7563-4db1-9011-2d313d9fb2d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:06.995541 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405fdd3e-7563-4db1-9011-2d313d9fb2d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:06.996213 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93686a-b87a-4389-9d47-7b25810aeb9a None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:06.996960 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405fdd3e-7563-4db1-9011-2d313d9fb2d0 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:06.997649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b93686a-b87a-4389-9d47-7b25810aeb9a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:06.999491 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 60/112] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:06 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:07.004488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93686a-b87a-4389-9d47-7b25810aeb9a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:07.011970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 53/113] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:07.012822 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fdd3e-7563-4db1-9011-2d313d9fb2d0 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:07.131761 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b93686a-b87a-4389-9d47-7b25810aeb9a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:07.131975 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fdd3e-7563-4db1-9011-2d313d9fb2d0 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:07.135997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:07.170788 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d77895-9275-4362-8797-2d019adb1de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:07.171634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d77895-9275-4362-8797-2d019adb1de9 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:07.171916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d77895-9275-4362-8797-2d019adb1de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:07.172191 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d77895-9275-4362-8797-2d019adb1de9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:07.173396 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c81e8f-ed44-4cf7-8784-2d0bb669b5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:07.174064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c81e8f-ed44-4cf7-8784-2d0bb669b5a1 None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:07.174346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c81e8f-ed44-4cf7-8784-2d0bb669b5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:07.174625 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c81e8f-ed44-4cf7-8784-2d0bb669b5a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:07.175435 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c81e8f-ed44-4cf7-8784-2d0bb669b5a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:07.176889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c81e8f-ed44-4cf7-8784-2d0bb669b5a1 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:07.182217 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d77895-9275-4362-8797-2d019adb1de9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:07.189768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-46d77895-9275-4362-8797-2d019adb1de9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:07.189768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:07.189768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:07.189768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:07.189768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:07.189768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:07.189768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:07.189768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:07.189768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:07.189768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:07.189768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:07.189768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:07.189768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:07.189768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:07.189768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:07.189768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:07.190847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:07.190847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:07.190847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:07.190847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:07.190847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:07.190847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:07.190847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:07.190847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:07.190847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:07.190847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:07.190847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:07.190847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:07.190847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:07.190847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:07.190847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:07.190847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:07.191708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:07.191708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:07.191708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:07.191708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:07.191708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:07.191708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:07.191708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:07.191708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:07.191708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:07.191708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:07.191708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:07.191708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:07.191708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 54/114] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:07.192406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebd88b-c7d2-425b-bf48-6e51ea6bd4ed None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:07.192615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebd88b-c7d2-425b-bf48-6e51ea6bd4ed None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:07.192816 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebd88b-c7d2-425b-bf48-6e51ea6bd4ed None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:07.194126 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 55/115] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:07.194936 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebd88b-c7d2-425b-bf48-6e51ea6bd4ed None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:07.213202 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 61/116] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jul 21 05:30:06 2022] POST /identity/v3/projects => generated 302 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:30:07.214084 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c81e8f-ed44-4cf7-8784-2d0bb669b5a1 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:07.258590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:07.321527 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebd88b-c7d2-425b-bf48-6e51ea6bd4ed None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:07.333791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c81e8f-ed44-4cf7-8784-2d0bb669b5a1 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:07.358397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f860c-dd88-4a9e-bc11-ca57df8636db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:07.359116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f860c-dd88-4a9e-bc11-ca57df8636db None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:07.359410 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f860c-dd88-4a9e-bc11-ca57df8636db None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:07.359695 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f860c-dd88-4a9e-bc11-ca57df8636db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:07.360552 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f860c-dd88-4a9e-bc11-ca57df8636db None admin] RBAC: Adding query filter params (role.id=784d41de5e1d415b93a1d4e0dc5c237b, scope.project.id=db74f3f0803a45359025fb277815314b, user.id=aa86fe483ab441bcbb42b09b4c81accc) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:07.360875 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f860c-dd88-4a9e-bc11-ca57df8636db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:07.362480 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f860c-dd88-4a9e-bc11-ca57df8636db None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:07.368845 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 56/117] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:07 2022] GET /identity/v3/role_assignments?user.id=aa86fe483ab441bcbb42b09b4c81accc&scope.project.id=db74f3f0803a45359025fb277815314b&role.id=784d41de5e1d415b93a1d4e0dc5c237b => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:07.373934 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c839c2-aca1-4288-8fd1-c6123ee8cb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:07.374952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c839c2-aca1-4288-8fd1-c6123ee8cb05 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:07.375276 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c839c2-aca1-4288-8fd1-c6123ee8cb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:07.376288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c839c2-aca1-4288-8fd1-c6123ee8cb05 None admin] PATH_INFO: `/v3/users/3b3cc4a53587419894e0f708d1e60e89` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:07.377154 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c839c2-aca1-4288-8fd1-c6123ee8cb05 None admin] RBAC: Authorizing `identity:get_user(user_id=3b3cc4a53587419894e0f708d1e60e89)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:07.378558 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c839c2-aca1-4288-8fd1-c6123ee8cb05 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:07.380578 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 62/118] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:07 2022] GET /identity/v3/users/3b3cc4a53587419894e0f708d1e60e89 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:07.380757 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c5657-d2ad-4f56-8b54-a8bc46c9d2da None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:07.381055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c5657-d2ad-4f56-8b54-a8bc46c9d2da None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:07.381205 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c839c2-aca1-4288-8fd1-c6123ee8cb05 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:07.381334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c5657-d2ad-4f56-8b54-a8bc46c9d2da None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:07.382690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 57/119] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:07 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:07.384558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d180c7-7b68-4e51-ac1c-0e7281133cbd None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:07.384558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d180c7-7b68-4e51-ac1c-0e7281133cbd None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:07.384558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d180c7-7b68-4e51-ac1c-0e7281133cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:07.405168 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-58d180c7-7b68-4e51-ac1c-0e7281133cbd None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:07.406394 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-58d180c7-7b68-4e51-ac1c-0e7281133cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:07.436319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:07.557179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 58/120] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:07.558639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77ca5a-1f36-4740-8995-166b47fea71b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:07.563368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c839c2-aca1-4288-8fd1-c6123ee8cb05 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:07.564421 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77ca5a-1f36-4740-8995-166b47fea71b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:07.564771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77ca5a-1f36-4740-8995-166b47fea71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:07.610906 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-df77ca5a-1f36-4740-8995-166b47fea71b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:07.612364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-df77ca5a-1f36-4740-8995-166b47fea71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:07.620712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3dba5b-3f1a-407e-9c3c-d9b419dd62cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:07.623042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3dba5b-3f1a-407e-9c3c-d9b419dd62cf None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:07.623042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3dba5b-3f1a-407e-9c3c-d9b419dd62cf None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:07.623042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3dba5b-3f1a-407e-9c3c-d9b419dd62cf None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:07.623624 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3dba5b-3f1a-407e-9c3c-d9b419dd62cf None admin] RBAC: Adding query filter params (name=member) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:07.623939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3dba5b-3f1a-407e-9c3c-d9b419dd62cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:07.626271 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3dba5b-3f1a-407e-9c3c-d9b419dd62cf None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:07.639217 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 63/121] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:07.640340 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3dba5b-3f1a-407e-9c3c-d9b419dd62cf None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:07.658731 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:07.749990 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 59/122] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:07.750848 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77ca5a-1f36-4740-8995-166b47fea71b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:07.795755 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:07.816422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3dba5b-3f1a-407e-9c3c-d9b419dd62cf None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:07.858676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dcfda-f13c-4c65-8deb-903719051ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:07.859658 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dcfda-f13c-4c65-8deb-903719051ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:07.859944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dcfda-f13c-4c65-8deb-903719051ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:07.860217 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dcfda-f13c-4c65-8deb-903719051ed0 None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:07.861445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dcfda-f13c-4c65-8deb-903719051ed0 None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:07.863067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dcfda-f13c-4c65-8deb-903719051ed0 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:07.864995 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 64/123] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:07 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:07.870622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dcfda-f13c-4c65-8deb-903719051ed0 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:07.888836 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77ca5a-1f36-4740-8995-166b47fea71b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:07.936389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210423c-ee5b-4bee-b0dd-b9b5be3888d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:07.937237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210423c-ee5b-4bee-b0dd-b9b5be3888d4 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:07.937525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210423c-ee5b-4bee-b0dd-b9b5be3888d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:07.937803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210423c-ee5b-4bee-b0dd-b9b5be3888d4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:07.950661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210423c-ee5b-4bee-b0dd-b9b5be3888d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:07.959077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-0210423c-ee5b-4bee-b0dd-b9b5be3888d4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:07.959077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:07.959077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:07.959077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:07.959077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:07.959077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:07.959077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:07.959077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:07.959077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:07.959077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:07.959077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:07.959077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:07.959077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:07.959077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:07.959077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:07.959077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:07.959953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:07.959953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:07.959953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:07.959953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:07.959953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:07.959953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:07.959953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:07.959953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:07.959953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:07.959953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:07.959953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:07.959953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:07.959953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:07.959953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:07.959953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:07.959953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:07.961341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:07.961341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:07.961341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:07.961341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:07.961341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:07.961341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:07.961341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:07.961341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:07.961341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:07.961341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:07.961341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:07.961341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:07.961341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 60/124] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:30:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:07.961341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210423c-ee5b-4bee-b0dd-b9b5be3888d4 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:07.995107 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:08.027074 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dcfda-f13c-4c65-8deb-903719051ed0 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:08.063422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe0e66-e1bc-4fe4-9cc0-cd868bd0a28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:08.064682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe0e66-e1bc-4fe4-9cc0-cd868bd0a28d None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:08.065161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe0e66-e1bc-4fe4-9cc0-cd868bd0a28d None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:08.065654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe0e66-e1bc-4fe4-9cc0-cd868bd0a28d None admin] PATH_INFO: `/v3/users/c23f5930d01c4d6f819cc9308da0b14e` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:08.067113 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe0e66-e1bc-4fe4-9cc0-cd868bd0a28d None admin] RBAC: Authorizing `identity:get_user(user_id=c23f5930d01c4d6f819cc9308da0b14e)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:08.068800 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe0e66-e1bc-4fe4-9cc0-cd868bd0a28d None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:08.071086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 65/125] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:07 2022] GET /identity/v3/users/c23f5930d01c4d6f819cc9308da0b14e => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:08.072230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe0e66-e1bc-4fe4-9cc0-cd868bd0a28d None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:08.081915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210423c-ee5b-4bee-b0dd-b9b5be3888d4 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:08.097966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:08.119152 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241f40e-a379-42e2-acd6-42a0210c5b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:08.120316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241f40e-a379-42e2-acd6-42a0210c5b68 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:08.121564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241f40e-a379-42e2-acd6-42a0210c5b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:08.121928 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241f40e-a379-42e2-acd6-42a0210c5b68 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:08.136419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241f40e-a379-42e2-acd6-42a0210c5b68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:08.144315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-2241f40e-a379-42e2-acd6-42a0210c5b68 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:08.144315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:08.144315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:08.144315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:08.144315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:08.144315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:08.144315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:08.144315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:08.144315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:08.144315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:08.144315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:08.144315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:08.144315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:08.144315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:08.144315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:08.144315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:08.145208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:08.145208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:08.145208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:08.145208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:08.145208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:08.145208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:08.145208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:08.145208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:08.145208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:08.145208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:08.145208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:08.145208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:08.145208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:08.145208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:08.145208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:08.145208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:08.146008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:08.146008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:08.146008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:08.146008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:08.146008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:08.146008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:08.146008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:08.146008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:08.146008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:08.146008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:08.146008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:08.146008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:08.150728 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 61/126] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:08.151689 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241f40e-a379-42e2-acd6-42a0210c5b68 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:08.183991 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe0e66-e1bc-4fe4-9cc0-cd868bd0a28d None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:08.219121 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ffb4e1-d20d-460e-b9ed-d5a4eeab95ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:08.220342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffb4e1-d20d-460e-b9ed-d5a4eeab95ae None admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:08.220834 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffb4e1-d20d-460e-b9ed-d5a4eeab95ae None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:08.221303 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ffb4e1-d20d-460e-b9ed-d5a4eeab95ae None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b/users/3b3cc4a53587419894e0f708d1e60e89/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:08.223503 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ffb4e1-d20d-460e-b9ed-d5a4eeab95ae None admin] RBAC: Authorizing `identity:create_grant(project_id=db74f3f0803a45359025fb277815314b, user_id=3b3cc4a53587419894e0f708d1e60e89, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:08.225289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ffb4e1-d20d-460e-b9ed-d5a4eeab95ae None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:08.250209 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:08.252239 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 66/127] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:30:08 2022] PUT /identity/v3/projects/db74f3f0803a45359025fb277815314b/users/3b3cc4a53587419894e0f708d1e60e89/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:30:08.253604 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ffb4e1-d20d-460e-b9ed-d5a4eeab95ae None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:08.283562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241f40e-a379-42e2-acd6-42a0210c5b68 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:08.352187 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:08.376829 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ffb4e1-d20d-460e-b9ed-d5a4eeab95ae None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:08.391853 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39060b3-2bdc-4ed7-a7f5-2304f35c1aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:08.392595 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39060b3-2bdc-4ed7-a7f5-2304f35c1aeb None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:08.392830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39060b3-2bdc-4ed7-a7f5-2304f35c1aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:08.393112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39060b3-2bdc-4ed7-a7f5-2304f35c1aeb None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:08.393902 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39060b3-2bdc-4ed7-a7f5-2304f35c1aeb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:08.394216 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39060b3-2bdc-4ed7-a7f5-2304f35c1aeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:08.395818 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39060b3-2bdc-4ed7-a7f5-2304f35c1aeb None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:08.419474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8f1a8-ea59-40fd-b4a0-3f0ae35dbf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:08.420797 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8f1a8-ea59-40fd-b4a0-3f0ae35dbf1a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:08.421296 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8f1a8-ea59-40fd-b4a0-3f0ae35dbf1a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:08.421799 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8f1a8-ea59-40fd-b4a0-3f0ae35dbf1a None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:08.423381 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8f1a8-ea59-40fd-b4a0-3f0ae35dbf1a None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:08.425146 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8f1a8-ea59-40fd-b4a0-3f0ae35dbf1a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:08.427786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 67/128] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:08 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:08.428900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8f1a8-ea59-40fd-b4a0-3f0ae35dbf1a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:08.429267 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 62/129] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:30:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:08.436090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39060b3-2bdc-4ed7-a7f5-2304f35c1aeb None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:08.542856 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8f1a8-ea59-40fd-b4a0-3f0ae35dbf1a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:08.580756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39060b3-2bdc-4ed7-a7f5-2304f35c1aeb None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:08.586749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf3b46-6fbc-4e2d-9acc-e2747a37a53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:08.588010 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf3b46-6fbc-4e2d-9acc-e2747a37a53f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:08.588502 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf3b46-6fbc-4e2d-9acc-e2747a37a53f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:08.588975 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf3b46-6fbc-4e2d-9acc-e2747a37a53f None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:08.590000 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf3b46-6fbc-4e2d-9acc-e2747a37a53f None admin] RBAC: Adding query filter params (name=member) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:08.590523 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf3b46-6fbc-4e2d-9acc-e2747a37a53f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:08.593129 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf3b46-6fbc-4e2d-9acc-e2747a37a53f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:08.603394 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 68/130] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:08.604644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf3b46-6fbc-4e2d-9acc-e2747a37a53f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:08.621232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:08.628533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534647f-593b-413a-9592-7f0c97d4d61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:08.629542 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534647f-593b-413a-9592-7f0c97d4d61f None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:08.629833 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534647f-593b-413a-9592-7f0c97d4d61f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:08.630114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534647f-593b-413a-9592-7f0c97d4d61f None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b/users/c23f5930d01c4d6f819cc9308da0b14e/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:08.632244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534647f-593b-413a-9592-7f0c97d4d61f None admin] RBAC: Authorizing `identity:create_grant(project_id=db74f3f0803a45359025fb277815314b, user_id=c23f5930d01c4d6f819cc9308da0b14e, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:08.633806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534647f-593b-413a-9592-7f0c97d4d61f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:08.653509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 63/131] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:30:08 2022] PUT /identity/v3/projects/db74f3f0803a45359025fb277815314b/users/c23f5930d01c4d6f819cc9308da0b14e/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:30:08.654514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534647f-593b-413a-9592-7f0c97d4d61f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:08.684241 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:08.727342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf3b46-6fbc-4e2d-9acc-e2747a37a53f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:08.780445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a25834-7322-4cdc-83f1-72e3cafe1bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:08.781833 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a25834-7322-4cdc-83f1-72e3cafe1bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:08.782416 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a25834-7322-4cdc-83f1-72e3cafe1bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:08.782894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a25834-7322-4cdc-83f1-72e3cafe1bc5 None admin] PATH_INFO: `/v3/users/c23f5930d01c4d6f819cc9308da0b14e` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:08.784313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a25834-7322-4cdc-83f1-72e3cafe1bc5 None admin] RBAC: Authorizing `identity:get_user(user_id=c23f5930d01c4d6f819cc9308da0b14e)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:08.791507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a25834-7322-4cdc-83f1-72e3cafe1bc5 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:08.795379 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 69/132] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:08 2022] GET /identity/v3/users/c23f5930d01c4d6f819cc9308da0b14e => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:08.801539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a25834-7322-4cdc-83f1-72e3cafe1bc5 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:08.846959 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534647f-593b-413a-9592-7f0c97d4d61f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:08.913683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0f54b-daf5-42e6-adb8-8e29b8121e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:08.914612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0f54b-daf5-42e6-adb8-8e29b8121e89 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:08.914893 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0f54b-daf5-42e6-adb8-8e29b8121e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:08.915292 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0f54b-daf5-42e6-adb8-8e29b8121e89 None admin] PATH_INFO: `/v3/users/c23f5930d01c4d6f819cc9308da0b14e` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:08.918030 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:08.920163 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0f54b-daf5-42e6-adb8-8e29b8121e89 None admin] RBAC: Authorizing `identity:get_user(user_id=c23f5930d01c4d6f819cc9308da0b14e)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:08.921467 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0f54b-daf5-42e6-adb8-8e29b8121e89 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:08.923790 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 64/133] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:08 2022] GET /identity/v3/users/c23f5930d01c4d6f819cc9308da0b14e => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:08.931132 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0f54b-daf5-42e6-adb8-8e29b8121e89 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:08.941208 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a25834-7322-4cdc-83f1-72e3cafe1bc5 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:08.978182 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa788bf-26c4-45fd-b31a-d50bb8c4cea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:08.979600 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa788bf-26c4-45fd-b31a-d50bb8c4cea8 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:08.980081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa788bf-26c4-45fd-b31a-d50bb8c4cea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:08.980549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa788bf-26c4-45fd-b31a-d50bb8c4cea8 None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:08.982002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa788bf-26c4-45fd-b31a-d50bb8c4cea8 None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:08.983777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa788bf-26c4-45fd-b31a-d50bb8c4cea8 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:08.985836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 70/134] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:08 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:08.992063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa788bf-26c4-45fd-b31a-d50bb8c4cea8 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:09.048123 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:09.072225 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0f54b-daf5-42e6-adb8-8e29b8121e89 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:09.127168 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30290df-2e2a-43ff-a2fc-cdcefb61ccf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:09.130525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30290df-2e2a-43ff-a2fc-cdcefb61ccf7 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:09.130815 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30290df-2e2a-43ff-a2fc-cdcefb61ccf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:09.131121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30290df-2e2a-43ff-a2fc-cdcefb61ccf7 None admin] PATH_INFO: `/v3/projects/d8f593f0a0934e5f9f2baf99016b98a7` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:09.133321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30290df-2e2a-43ff-a2fc-cdcefb61ccf7 None admin] RBAC: Authorizing `identity:get_project(project_id=d8f593f0a0934e5f9f2baf99016b98a7)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:09.135898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30290df-2e2a-43ff-a2fc-cdcefb61ccf7 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:09.137728 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 65/135] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:08 2022] GET /identity/v3/projects/d8f593f0a0934e5f9f2baf99016b98a7 => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:09.146151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30290df-2e2a-43ff-a2fc-cdcefb61ccf7 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:09.203245 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa788bf-26c4-45fd-b31a-d50bb8c4cea8 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:09.260814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828bd6bf-33e1-466c-9bf6-b94edd8ec28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:09.261786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828bd6bf-33e1-466c-9bf6-b94edd8ec28f None admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:09.262075 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828bd6bf-33e1-466c-9bf6-b94edd8ec28f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:09.262526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828bd6bf-33e1-466c-9bf6-b94edd8ec28f None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b/users/c23f5930d01c4d6f819cc9308da0b14e/roles/a43ba93f358b4b1a8a445851a5a216a0` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:09.265043 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828bd6bf-33e1-466c-9bf6-b94edd8ec28f None admin] RBAC: Authorizing `identity:create_grant(project_id=db74f3f0803a45359025fb277815314b, user_id=c23f5930d01c4d6f819cc9308da0b14e, role_id=a43ba93f358b4b1a8a445851a5a216a0)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:09.266545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828bd6bf-33e1-466c-9bf6-b94edd8ec28f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:09.298266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 71/136] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:30:08 2022] PUT /identity/v3/projects/db74f3f0803a45359025fb277815314b/users/c23f5930d01c4d6f819cc9308da0b14e/roles/a43ba93f358b4b1a8a445851a5a216a0 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:30:09.385677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30290df-2e2a-43ff-a2fc-cdcefb61ccf7 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:09.461275 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f850c6a1-42c7-4b57-b3c2-c31fe4aab530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:09.462239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850c6a1-42c7-4b57-b3c2-c31fe4aab530 None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:09.462526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850c6a1-42c7-4b57-b3c2-c31fe4aab530 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:09.462802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f850c6a1-42c7-4b57-b3c2-c31fe4aab530 None admin] PATH_INFO: `/v3/projects/d8f593f0a0934e5f9f2baf99016b98a7/users/c23f5930d01c4d6f819cc9308da0b14e/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:09.468363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850c6a1-42c7-4b57-b3c2-c31fe4aab530 None admin] RBAC: Authorizing `identity:create_grant(project_id=d8f593f0a0934e5f9f2baf99016b98a7, user_id=c23f5930d01c4d6f819cc9308da0b14e, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:09.469890 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f850c6a1-42c7-4b57-b3c2-c31fe4aab530 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:09.504411 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 66/137] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:30:09 2022] PUT /identity/v3/projects/d8f593f0a0934e5f9f2baf99016b98a7/users/c23f5930d01c4d6f819cc9308da0b14e/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:30:09.933642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710345c-e86f-4716-bae6-dc6733fbab4d None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:09.934148 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710345c-e86f-4716-bae6-dc6733fbab4d None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:09.934148 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710345c-e86f-4716-bae6-dc6733fbab4d None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:09.935719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 72/138] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:09.950868 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17abb28-d431-4dd0-ad58-e205d064eedd None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:09.951386 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17abb28-d431-4dd0-ad58-e205d064eedd None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:09.951473 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17abb28-d431-4dd0-ad58-e205d064eedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:09.965759 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:09.971259 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-f17abb28-d431-4dd0-ad58-e205d064eedd None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:09.972438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f17abb28-d431-4dd0-ad58-e205d064eedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:10.013121 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225418c-629c-4a44-8452-6947df984d2b None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:10.013458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225418c-629c-4a44-8452-6947df984d2b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:10.013671 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225418c-629c-4a44-8452-6947df984d2b None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:10.015081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 73/139] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:30:10.020571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fb057-1e4d-47f4-a87d-ee86106c53f9 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:10.020848 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fb057-1e4d-47f4-a87d-ee86106c53f9 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:10.021314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fb057-1e4d-47f4-a87d-ee86106c53f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:10.061325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-e31fb057-1e4d-47f4-a87d-ee86106c53f9 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:10.062549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-e31fb057-1e4d-47f4-a87d-ee86106c53f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:10.065390 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:10.066439 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 67/140] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:10.104576 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17abb28-d431-4dd0-ad58-e205d064eedd None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:10.188493 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 74/141] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:10.255768 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fb057-1e4d-47f4-a87d-ee86106c53f9 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:10.256100 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17abb28-d431-4dd0-ad58-e205d064eedd None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:10.298106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6aef1-38d3-43e5-989c-89a63bb221ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:10.299272 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6aef1-38d3-43e5-989c-89a63bb221ac None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:10.299580 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6aef1-38d3-43e5-989c-89a63bb221ac None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:10.300150 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6aef1-38d3-43e5-989c-89a63bb221ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:10.301521 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6aef1-38d3-43e5-989c-89a63bb221ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:10.303528 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6aef1-38d3-43e5-989c-89a63bb221ac None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:10.305405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 68/142] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:30:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:10.310740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6aef1-38d3-43e5-989c-89a63bb221ac None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:10.314924 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:10.425894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6aef1-38d3-43e5-989c-89a63bb221ac None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:10.466779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ae756-9197-4190-ade3-23acc712dbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:10.467435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ae756-9197-4190-ade3-23acc712dbe3 None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:10.467725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ae756-9197-4190-ade3-23acc712dbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:10.468005 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ae756-9197-4190-ade3-23acc712dbe3 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:10.471037 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ae756-9197-4190-ade3-23acc712dbe3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:10.471037 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ae756-9197-4190-ade3-23acc712dbe3 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:10.475033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fb057-1e4d-47f4-a87d-ee86106c53f9 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:10.543172 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 69/143] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:30:10 2022] POST /identity/v3/users => generated 283 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:30:10.544670 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b0c39-5049-4382-8778-4dcf486142c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:10.545525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b0c39-5049-4382-8778-4dcf486142c6 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:10.545810 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b0c39-5049-4382-8778-4dcf486142c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:10.546091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b0c39-5049-4382-8778-4dcf486142c6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:10.560276 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:10.564324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b0c39-5049-4382-8778-4dcf486142c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:10.572804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-6d5b0c39-5049-4382-8778-4dcf486142c6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:10.572804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:10.572804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:10.572804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:10.572804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:10.572804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:10.572804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:10.572804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:10.572804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:10.572804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:10.572804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:10.572804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:10.572804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:10.572804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:10.572804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:10.572804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:10.573717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:10.573717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:10.573717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:10.573717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:10.573717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:10.573717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:10.573717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:10.573717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:10.573717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:10.573717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:10.573717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:10.573717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:10.573717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:10.573717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:10.573717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:10.573717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:10.574488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:10.574488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:10.574488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:10.574488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:10.574488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:10.574488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:10.574488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:10.574488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:10.574488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:10.574488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:10.574488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:10.574488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:10.574488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 75/144] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:10.584020 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ae756-9197-4190-ade3-23acc712dbe3 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:10.758846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:10.832822 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ae756-9197-4190-ade3-23acc712dbe3 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:10.884728 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce71431-6d95-4861-abda-ee9e7997ef21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:10.885431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce71431-6d95-4861-abda-ee9e7997ef21 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:10.885725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce71431-6d95-4861-abda-ee9e7997ef21 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:10.886009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce71431-6d95-4861-abda-ee9e7997ef21 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:10.886780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce71431-6d95-4861-abda-ee9e7997ef21 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:10.887130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce71431-6d95-4861-abda-ee9e7997ef21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:10.888602 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce71431-6d95-4861-abda-ee9e7997ef21 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:10.902539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 70/145] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:10.908214 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b0c39-5049-4382-8778-4dcf486142c6 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:11.027142 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b0c39-5049-4382-8778-4dcf486142c6 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:11.064160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0f68c-5a5c-419c-b040-53f063f9cf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:11.065320 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0f68c-5a5c-419c-b040-53f063f9cf73 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:11.065705 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0f68c-5a5c-419c-b040-53f063f9cf73 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:11.066079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0f68c-5a5c-419c-b040-53f063f9cf73 None admin] PATH_INFO: `/v3/users/aa86fe483ab441bcbb42b09b4c81accc` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:11.067440 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0f68c-5a5c-419c-b040-53f063f9cf73 None admin] RBAC: Authorizing `identity:get_user(user_id=aa86fe483ab441bcbb42b09b4c81accc)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:11.068925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0f68c-5a5c-419c-b040-53f063f9cf73 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:11.071179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 76/146] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:10 2022] GET /identity/v3/users/aa86fe483ab441bcbb42b09b4c81accc => generated 289 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:11.076843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce71431-6d95-4861-abda-ee9e7997ef21 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:11.200922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558995d-ddaa-4a6b-ada0-1702d0c515c8 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:11.201253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558995d-ddaa-4a6b-ada0-1702d0c515c8 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:11.201486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558995d-ddaa-4a6b-ada0-1702d0c515c8 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:11.203149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 77/147] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:11.210028 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6a774-7c1e-4133-abf4-ea09c25dd926 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:11.210333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6a774-7c1e-4133-abf4-ea09c25dd926 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:11.210589 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6a774-7c1e-4133-abf4-ea09c25dd926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:11.240772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce71431-6d95-4861-abda-ee9e7997ef21 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:11.245968 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-4ae6a774-7c1e-4133-abf4-ea09c25dd926 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:11.247338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4ae6a774-7c1e-4133-abf4-ea09c25dd926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:11.287124 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:11.301063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7262b22-1278-446d-a7f2-bd7baddaa1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:11.302268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7262b22-1278-446d-a7f2-bd7baddaa1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:11.302748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7262b22-1278-446d-a7f2-bd7baddaa1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:11.303257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7262b22-1278-446d-a7f2-bd7baddaa1e4 None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:11.306240 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7262b22-1278-446d-a7f2-bd7baddaa1e4 None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:11.307937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7262b22-1278-446d-a7f2-bd7baddaa1e4 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:11.310179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 71/148] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:11 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:11.318153 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7262b22-1278-446d-a7f2-bd7baddaa1e4 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:11.332519 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:11.337272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 78/149] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:11.388721 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6a774-7c1e-4133-abf4-ea09c25dd926 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:11.512476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7262b22-1278-446d-a7f2-bd7baddaa1e4 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:11.566241 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6a774-7c1e-4133-abf4-ea09c25dd926 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:11.579797 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90c5af-16e5-4e7a-b9ca-784126ef95f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:11.581178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90c5af-16e5-4e7a-b9ca-784126ef95f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:11.582068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90c5af-16e5-4e7a-b9ca-784126ef95f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:11.582562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90c5af-16e5-4e7a-b9ca-784126ef95f9 None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b/users/aa86fe483ab441bcbb42b09b4c81accc/roles/784d41de5e1d415b93a1d4e0dc5c237b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:11.585554 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90c5af-16e5-4e7a-b9ca-784126ef95f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=db74f3f0803a45359025fb277815314b, user_id=aa86fe483ab441bcbb42b09b4c81accc, role_id=784d41de5e1d415b93a1d4e0dc5c237b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:11.587213 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90c5af-16e5-4e7a-b9ca-784126ef95f9 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:11.610761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 72/150] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:30:11 2022] PUT /identity/v3/projects/db74f3f0803a45359025fb277815314b/users/aa86fe483ab441bcbb42b09b4c81accc/roles/784d41de5e1d415b93a1d4e0dc5c237b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:30:11.642626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33ac89-7973-445c-b0d9-dbd85069d434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:11.643509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33ac89-7973-445c-b0d9-dbd85069d434 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:11.643795 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33ac89-7973-445c-b0d9-dbd85069d434 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:11.644073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33ac89-7973-445c-b0d9-dbd85069d434 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:11.671836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33ac89-7973-445c-b0d9-dbd85069d434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:11.680212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-ab33ac89-7973-445c-b0d9-dbd85069d434 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:11.680212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:11.680212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:11.680212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:11.680212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:11.680212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:11.680212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:11.680212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:11.680212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:11.680212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:11.680212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:11.680212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:11.680212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:11.680212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:11.680212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:11.680212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:11.682527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:11.682527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:11.682527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:11.682527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:11.682527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:11.682527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:11.682527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:11.682527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:11.682527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:11.682527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:11.682527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:11.682527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:11.682527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:11.682527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:11.682527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:11.682527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:11.684599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:11.684599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:11.684599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:11.684599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:11.684599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:11.684599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:11.684599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:11.684599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:11.684599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:11.684599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:11.684599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:11.684599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:11.684599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 79/151] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:30:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:11.688280 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90c5af-16e5-4e7a-b9ca-784126ef95f9 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:11.774610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:11.847677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90c5af-16e5-4e7a-b9ca-784126ef95f9 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:11.887888 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f42ec-a921-45e6-b0c7-85cb134579a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:11.888606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f42ec-a921-45e6-b0c7-85cb134579a6 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:11.888893 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f42ec-a921-45e6-b0c7-85cb134579a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:11.889167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f42ec-a921-45e6-b0c7-85cb134579a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:11.889958 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f42ec-a921-45e6-b0c7-85cb134579a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:11.890265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f42ec-a921-45e6-b0c7-85cb134579a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:11.891800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f42ec-a921-45e6-b0c7-85cb134579a6 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:11.902999 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 73/152] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:11.912204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33ac89-7973-445c-b0d9-dbd85069d434 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:12.071630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:12.126502 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33ac89-7973-445c-b0d9-dbd85069d434 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:12.172442 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3e6f7-740c-43c5-b139-0d6853dbe669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:12.172828 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3e6f7-740c-43c5-b139-0d6853dbe669 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:12.172948 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3e6f7-740c-43c5-b139-0d6853dbe669 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:12.173802 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3e6f7-740c-43c5-b139-0d6853dbe669 None admin] PATH_INFO: `/v3/users/3b3cc4a53587419894e0f708d1e60e89` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:12.183061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3e6f7-740c-43c5-b139-0d6853dbe669 None admin] RBAC: Authorizing `identity:get_user(user_id=3b3cc4a53587419894e0f708d1e60e89)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:12.183061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3e6f7-740c-43c5-b139-0d6853dbe669 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:12.183363 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 80/153] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:11 2022] GET /identity/v3/users/3b3cc4a53587419894e0f708d1e60e89 => generated 247 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:12.192230 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f42ec-a921-45e6-b0c7-85cb134579a6 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:12.428768 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de409f4b-80ca-426e-9a18-9753b271ca85 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:12.429152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de409f4b-80ca-426e-9a18-9753b271ca85 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:12.429328 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de409f4b-80ca-426e-9a18-9753b271ca85 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:12.430861 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 81/154] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:12.441499 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56f376-c620-439d-8540-ce07d20b3903 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:12.441499 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56f376-c620-439d-8540-ce07d20b3903 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:12.441499 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56f376-c620-439d-8540-ce07d20b3903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:12.465619 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f42ec-a921-45e6-b0c7-85cb134579a6 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:12.492021 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:12.508949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-9c56f376-c620-439d-8540-ce07d20b3903 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:12.509495 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9c56f376-c620-439d-8540-ce07d20b3903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:12.539849 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964adaba-97c7-4ba7-830a-aa32b5c929a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:12.540705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964adaba-97c7-4ba7-830a-aa32b5c929a2 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:12.541170 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964adaba-97c7-4ba7-830a-aa32b5c929a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:12.541447 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964adaba-97c7-4ba7-830a-aa32b5c929a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:12.687538 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964adaba-97c7-4ba7-830a-aa32b5c929a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:12.689017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964adaba-97c7-4ba7-830a-aa32b5c929a2 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:12.833658 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 82/155] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:12.835326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c4f92-a65f-44f5-861a-215c448607b3 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:12.835737 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c4f92-a65f-44f5-861a-215c448607b3 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:12.836155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c4f92-a65f-44f5-861a-215c448607b3 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:12.837666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 83/156] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:12.838833 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040b35c-0a3d-415a-b93a-90b0bf23b7c7 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:12.839270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040b35c-0a3d-415a-b93a-90b0bf23b7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:12.839676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040b35c-0a3d-415a-b93a-90b0bf23b7c7 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:12.841004 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 84/157] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:12.848270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415d1fe-fb4e-412f-be4c-95ac05ab7550 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:12.848719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415d1fe-fb4e-412f-be4c-95ac05ab7550 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:12.849113 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415d1fe-fb4e-412f-be4c-95ac05ab7550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:12.916097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-8415d1fe-fb4e-412f-be4c-95ac05ab7550 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:12.917530 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8415d1fe-fb4e-412f-be4c-95ac05ab7550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:12.940122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 74/158] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 745 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:12.947102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7c0db-b682-4170-a57c-e51d2d8c776b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:12.947102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7c0db-b682-4170-a57c-e51d2d8c776b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:12.947102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7c0db-b682-4170-a57c-e51d2d8c776b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:12.979040 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-09b7c0db-b682-4170-a57c-e51d2d8c776b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:12.979040 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-09b7c0db-b682-4170-a57c-e51d2d8c776b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:13.005412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:13.165731 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 75/159] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:13.171685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7c0db-b682-4170-a57c-e51d2d8c776b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:13.195704 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:13.289144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 85/160] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:13.289144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415d1fe-fb4e-412f-be4c-95ac05ab7550 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:13.610658 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7c0db-b682-4170-a57c-e51d2d8c776b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:13.667684 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415d1fe-fb4e-412f-be4c-95ac05ab7550 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:13.780162 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae2d11-1115-41f1-add9-af2bc4b271e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:13.781454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae2d11-1115-41f1-add9-af2bc4b271e2 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:13.781963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae2d11-1115-41f1-add9-af2bc4b271e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:13.782450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae2d11-1115-41f1-add9-af2bc4b271e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:13.806138 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0c61d-c223-494c-9fad-2debb023f584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:13.808175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0c61d-c223-494c-9fad-2debb023f584 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:13.808175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0c61d-c223-494c-9fad-2debb023f584 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:13.808175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0c61d-c223-494c-9fad-2debb023f584 None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:13.809112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0c61d-c223-494c-9fad-2debb023f584 None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:13.811060 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0c61d-c223-494c-9fad-2debb023f584 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:13.820653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 86/161] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:13 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 533 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:13.821479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0c61d-c223-494c-9fad-2debb023f584 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:13.839729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae2d11-1115-41f1-add9-af2bc4b271e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:13.855151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-d9ae2d11-1115-41f1-add9-af2bc4b271e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:13.855151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:13.855151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:13.855151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:13.855151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:13.855151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:13.855151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:13.855151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:13.855151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:13.855151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:13.855151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:13.855151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:13.855151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:13.855151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:13.855151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:13.855151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:13.856094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:13.856094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:13.856094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:13.856094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:13.856094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:13.856094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:13.856094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:13.856094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:13.856094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:13.856094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:13.856094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:13.856094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:13.856094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:13.856094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:13.856094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:13.856094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:13.856937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:13.856937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:13.856937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:13.856937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:13.856937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:13.856937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:13.856937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:13.856937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:13.856937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:13.856937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:13.856937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:13.856937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:13.864996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 76/162] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 699 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:13.866133 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae2d11-1115-41f1-add9-af2bc4b271e2 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:13.872610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:13.945536 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:14.055144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0c61d-c223-494c-9fad-2debb023f584 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:14.137548 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae2d11-1115-41f1-add9-af2bc4b271e2 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:14.200299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91874f67-bcd3-46f8-8f8a-81745f6fd6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:14.201141 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91874f67-bcd3-46f8-8f8a-81745f6fd6cf None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:14.201418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91874f67-bcd3-46f8-8f8a-81745f6fd6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:14.201685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91874f67-bcd3-46f8-8f8a-81745f6fd6cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:14.212482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91874f67-bcd3-46f8-8f8a-81745f6fd6cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:14.220269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e064ce5-9b6f-4787-9137-834bbd506272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:14.220737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-91874f67-bcd3-46f8-8f8a-81745f6fd6cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:14.220737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:14.220737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:14.220737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:14.220737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:14.220737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:14.220737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:14.220737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:14.220737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:14.220737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:14.220737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:14.220737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:14.220737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:14.220737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:14.220737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:14.220737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:14.221590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:14.221590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:14.221590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:14.221590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:14.221590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:14.221590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:14.221590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:14.221590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:14.221590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:14.221590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:14.221590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:14.221590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:14.221590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:14.221590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:14.221590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:14.221590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:14.222354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:14.222354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:14.222354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:14.222354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:14.222354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:14.222354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:14.222354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:14.222354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:14.222354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:14.222354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:14.222354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:14.222354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:14.222951 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e064ce5-9b6f-4787-9137-834bbd506272 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:14.222951 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e064ce5-9b6f-4787-9137-834bbd506272 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:14.222951 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e064ce5-9b6f-4787-9137-834bbd506272 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:14.223153 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 77/163] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:30:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:14.223153 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91874f67-bcd3-46f8-8f8a-81745f6fd6cf None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:14.238322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e064ce5-9b6f-4787-9137-834bbd506272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:14.246224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-1e064ce5-9b6f-4787-9137-834bbd506272 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:14.246224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:14.246224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:14.246224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:14.246224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:14.246224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:14.246224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:14.246224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:14.246224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:14.246224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:14.246224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:14.246224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:14.246224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:14.246224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:14.246224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:14.246224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:14.247373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:14.247373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:14.247373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:14.247373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:14.247373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:14.247373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:14.247373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:14.247373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:14.247373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:14.247373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:14.247373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:14.247373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:14.247373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:14.247373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:14.247373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:14.247373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:14.248167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:14.248167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:14.248167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:14.248167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:14.248167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:14.248167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:14.248167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:14.248167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:14.248167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:14.248167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:14.248167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:14.248167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:14.248167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 87/164] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 426 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:14.248830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e064ce5-9b6f-4787-9137-834bbd506272 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:14.354866 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e064ce5-9b6f-4787-9137-834bbd506272 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:14.371958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:14.390384 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1ad38-981f-483c-a2a3-6a13a57948ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:14.391163 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1ad38-981f-483c-a2a3-6a13a57948ce None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:14.391571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1ad38-981f-483c-a2a3-6a13a57948ce None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:14.391952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1ad38-981f-483c-a2a3-6a13a57948ce None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:14.392928 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1ad38-981f-483c-a2a3-6a13a57948ce None admin] RBAC: Adding query filter params (name=member) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:14.393338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1ad38-981f-483c-a2a3-6a13a57948ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:14.394857 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1ad38-981f-483c-a2a3-6a13a57948ce None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:14.403786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 88/165] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:14.406450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7717938-95f8-40ba-8e4e-a160149fed5a None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:14.406882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7717938-95f8-40ba-8e4e-a160149fed5a None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:14.407304 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7717938-95f8-40ba-8e4e-a160149fed5a None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:14.408672 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 89/166] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:14.411360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7717938-95f8-40ba-8e4e-a160149fed5a None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:14.444484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:14.508058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91874f67-bcd3-46f8-8f8a-81745f6fd6cf None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:14.544641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7717938-95f8-40ba-8e4e-a160149fed5a None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:14.580792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6344b96-9842-4a89-9eb7-2bc5ded202a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:14.582078 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47c6c1-e9ba-4f89-bdc9-b5ab0585af29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:14.582754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47c6c1-e9ba-4f89-bdc9-b5ab0585af29 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:14.583056 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47c6c1-e9ba-4f89-bdc9-b5ab0585af29 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:14.583333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47c6c1-e9ba-4f89-bdc9-b5ab0585af29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:14.584183 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47c6c1-e9ba-4f89-bdc9-b5ab0585af29 None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=db74f3f0803a45359025fb277815314b, user.id=3b3cc4a53587419894e0f708d1e60e89) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:14.584498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47c6c1-e9ba-4f89-bdc9-b5ab0585af29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:14.585409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6344b96-9842-4a89-9eb7-2bc5ded202a5 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:14.585696 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6344b96-9842-4a89-9eb7-2bc5ded202a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:14.585967 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6344b96-9842-4a89-9eb7-2bc5ded202a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:14.586715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6344b96-9842-4a89-9eb7-2bc5ded202a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:14.587116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6344b96-9842-4a89-9eb7-2bc5ded202a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:14.588526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6344b96-9842-4a89-9eb7-2bc5ded202a5 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:14.590279 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47c6c1-e9ba-4f89-bdc9-b5ab0585af29 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:14.602395 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 90/167] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:30:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:14.605057 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 78/168] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:14 2022] GET /identity/v3/role_assignments?user.id=3b3cc4a53587419894e0f708d1e60e89&scope.project.id=db74f3f0803a45359025fb277815314b&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 608 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:14.605898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47c6c1-e9ba-4f89-bdc9-b5ab0585af29 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:14.606902 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6344b96-9842-4a89-9eb7-2bc5ded202a5 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:14.683548 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:14.857947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6344b96-9842-4a89-9eb7-2bc5ded202a5 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:14.912049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47c6c1-e9ba-4f89-bdc9-b5ab0585af29 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:14.939849 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:14.944765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92667f-e5d6-48db-ab7d-01478526f468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:14.945468 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92667f-e5d6-48db-ab7d-01478526f468 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:14.945751 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92667f-e5d6-48db-ab7d-01478526f468 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:14.946029 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92667f-e5d6-48db-ab7d-01478526f468 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:14.946852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92667f-e5d6-48db-ab7d-01478526f468 None admin] RBAC: Adding query filter params (name=member) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:14.947194 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92667f-e5d6-48db-ab7d-01478526f468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:14.948596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92667f-e5d6-48db-ab7d-01478526f468 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:14.949895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd41fb-720e-42ad-9a49-15a720dbf099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:14.950785 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd41fb-720e-42ad-9a49-15a720dbf099 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:14.951102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd41fb-720e-42ad-9a49-15a720dbf099 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:14.951381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd41fb-720e-42ad-9a49-15a720dbf099 None admin] PATH_INFO: `/v3/users/c23f5930d01c4d6f819cc9308da0b14e` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:14.952590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd41fb-720e-42ad-9a49-15a720dbf099 None admin] RBAC: Authorizing `identity:get_user(user_id=c23f5930d01c4d6f819cc9308da0b14e)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:14.954056 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd41fb-720e-42ad-9a49-15a720dbf099 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:14.955739 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 79/169] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:14 2022] GET /identity/v3/users/c23f5930d01c4d6f819cc9308da0b14e => generated 275 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:14.956905 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28cf1d-922e-42ea-94e6-b296b59b9c45 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:14.957190 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28cf1d-922e-42ea-94e6-b296b59b9c45 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:14.957468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28cf1d-922e-42ea-94e6-b296b59b9c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:14.968073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 91/170] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:14.968382 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92667f-e5d6-48db-ab7d-01478526f468 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:14.980989 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-0a28cf1d-922e-42ea-94e6-b296b59b9c45 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:14.982199 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-0a28cf1d-922e-42ea-94e6-b296b59b9c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:15.061309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 80/171] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:15.062224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28cf1d-922e-42ea-94e6-b296b59b9c45 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:15.105843 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:15.116069 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92667f-e5d6-48db-ab7d-01478526f468 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:15.158865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec27b6-3a36-4a3b-ae85-671c458670ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:15.159218 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec27b6-3a36-4a3b-ae85-671c458670ca None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:15.159460 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec27b6-3a36-4a3b-ae85-671c458670ca None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:15.162104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec27b6-3a36-4a3b-ae85-671c458670ca None admin] PATH_INFO: `/v3/users/c23f5930d01c4d6f819cc9308da0b14e` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:15.162104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec27b6-3a36-4a3b-ae85-671c458670ca None admin] RBAC: Authorizing `identity:get_user(user_id=c23f5930d01c4d6f819cc9308da0b14e)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:15.162486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec27b6-3a36-4a3b-ae85-671c458670ca None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:15.165968 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 92/172] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:14 2022] GET /identity/v3/users/c23f5930d01c4d6f819cc9308da0b14e => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:15.171326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec27b6-3a36-4a3b-ae85-671c458670ca None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:15.178305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28cf1d-922e-42ea-94e6-b296b59b9c45 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:15.207240 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:15.226255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dc17b-e099-46e3-90d8-31944ccfdc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:15.226898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dc17b-e099-46e3-90d8-31944ccfdc39 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:15.227218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dc17b-e099-46e3-90d8-31944ccfdc39 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:15.227552 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dc17b-e099-46e3-90d8-31944ccfdc39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:15.287344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec27b6-3a36-4a3b-ae85-671c458670ca None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:15.303093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29dc17b-e099-46e3-90d8-31944ccfdc39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:15.304578 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29dc17b-e099-46e3-90d8-31944ccfdc39 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:15.333100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66ca1f-2371-48af-8596-1a30c5e8797b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:15.333993 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66ca1f-2371-48af-8596-1a30c5e8797b None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:15.334280 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66ca1f-2371-48af-8596-1a30c5e8797b None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:15.334558 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66ca1f-2371-48af-8596-1a30c5e8797b None admin] PATH_INFO: `/v3/users/c23f5930d01c4d6f819cc9308da0b14e` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:15.335775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66ca1f-2371-48af-8596-1a30c5e8797b None admin] RBAC: Authorizing `identity:get_user(user_id=c23f5930d01c4d6f819cc9308da0b14e)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:15.337318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66ca1f-2371-48af-8596-1a30c5e8797b None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:15.338974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 93/173] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:15 2022] GET /identity/v3/users/c23f5930d01c4d6f819cc9308da0b14e => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:15.340180 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa85b2-591f-4231-86e2-f529f8832557 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:15.340462 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa85b2-591f-4231-86e2-f529f8832557 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:15.340739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa85b2-591f-4231-86e2-f529f8832557 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:15.342129 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 94/174] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:15.343777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa85b2-591f-4231-86e2-f529f8832557 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:15.381681 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:15.415043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 81/175] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:15.416767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dc17b-e099-46e3-90d8-31944ccfdc39 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:15.442837 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:15.451473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa85b2-591f-4231-86e2-f529f8832557 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:15.484401 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1e10e-c21d-41ff-9d0f-c6710438acfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:15.485129 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1e10e-c21d-41ff-9d0f-c6710438acfa None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:15.485408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1e10e-c21d-41ff-9d0f-c6710438acfa None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:15.485680 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1e10e-c21d-41ff-9d0f-c6710438acfa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:15.495389 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb1e10e-c21d-41ff-9d0f-c6710438acfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:15.502880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-bdb1e10e-c21d-41ff-9d0f-c6710438acfa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:15.502880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:15.502880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:15.502880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:15.502880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:15.502880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:15.502880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:15.502880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:15.502880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:15.502880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:15.502880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:15.502880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:15.502880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:15.502880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:15.502880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:15.502880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:15.503926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:15.503926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:15.503926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:15.503926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:15.503926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:15.503926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:15.503926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:15.503926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:15.503926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:15.503926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:15.503926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:15.503926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:15.503926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:15.503926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:15.503926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:15.503926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:15.504712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:15.504712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:15.504712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:15.504712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:15.504712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:15.504712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:15.504712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:15.504712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:15.504712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:15.504712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:15.504712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:15.504712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:15.504712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 95/176] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:30:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:15.508271 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07baf1b-5f5b-4b51-a2fe-c89cac611ebc None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:15.508590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07baf1b-5f5b-4b51-a2fe-c89cac611ebc None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:15.509007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07baf1b-5f5b-4b51-a2fe-c89cac611ebc None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:15.510366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 96/177] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:15.511039 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07baf1b-5f5b-4b51-a2fe-c89cac611ebc None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:15.515856 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dc17b-e099-46e3-90d8-31944ccfdc39 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:15.550843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de43e7-a5a0-4657-aa46-7e095772859b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:15.551536 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de43e7-a5a0-4657-aa46-7e095772859b None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:15.551821 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de43e7-a5a0-4657-aa46-7e095772859b None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:15.552100 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de43e7-a5a0-4657-aa46-7e095772859b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:15.603042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:15.611325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07baf1b-5f5b-4b51-a2fe-c89cac611ebc None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:15.617724 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de43e7-a5a0-4657-aa46-7e095772859b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:15.619711 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de43e7-a5a0-4657-aa46-7e095772859b None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:15.647468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:15.654496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134423c3-0656-4586-984b-a9162b80e272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:15.655230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134423c3-0656-4586-984b-a9162b80e272 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:15.655518 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134423c3-0656-4586-984b-a9162b80e272 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:15.655799 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134423c3-0656-4586-984b-a9162b80e272 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:15.733167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134423c3-0656-4586-984b-a9162b80e272 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:15.734719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134423c3-0656-4586-984b-a9162b80e272 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:15.740177 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 82/178] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:15.741487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378ee80-8e7c-4157-95ec-d6966b208972 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:15.741774 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378ee80-8e7c-4157-95ec-d6966b208972 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:15.742048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378ee80-8e7c-4157-95ec-d6966b208972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:15.757898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-5378ee80-8e7c-4157-95ec-d6966b208972 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:15.759215 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5378ee80-8e7c-4157-95ec-d6966b208972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:15.826147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:15.838404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 97/179] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:15.838533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 83/180] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:15.838637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c6baf-9204-44f5-837f-9749efdb0b43 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:15.838637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c6baf-9204-44f5-837f-9749efdb0b43 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:15.838637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c6baf-9204-44f5-837f-9749efdb0b43 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:15.838816 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378ee80-8e7c-4157-95ec-d6966b208972 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:15.838882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 98/181] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:15.841125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c6baf-9204-44f5-837f-9749efdb0b43 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:15.896386 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:15.940706 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378ee80-8e7c-4157-95ec-d6966b208972 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:15.941306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c6baf-9204-44f5-837f-9749efdb0b43 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:15.974577 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f93b5d-3cfd-4372-bf8e-a4201997cb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:15.975192 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3f41b-144c-4243-ba8b-c18e7680d3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:15.975488 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f93b5d-3cfd-4372-bf8e-a4201997cb20 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:15.975769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f93b5d-3cfd-4372-bf8e-a4201997cb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:15.975862 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3f41b-144c-4243-ba8b-c18e7680d3ad None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:15.976042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f93b5d-3cfd-4372-bf8e-a4201997cb20 None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:15.976160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3f41b-144c-4243-ba8b-c18e7680d3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:15.976407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d3f41b-144c-4243-ba8b-c18e7680d3ad None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:15.977163 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d3f41b-144c-4243-ba8b-c18e7680d3ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:15.977249 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f93b5d-3cfd-4372-bf8e-a4201997cb20 None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:15.977505 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d3f41b-144c-4243-ba8b-c18e7680d3ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:15.978691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f93b5d-3cfd-4372-bf8e-a4201997cb20 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:15.978882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d3f41b-144c-4243-ba8b-c18e7680d3ad None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:15.985941 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 84/182] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:15 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:15.985941 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002d1f3-9222-4676-9b78-c37b1be0e721 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:15.985941 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002d1f3-9222-4676-9b78-c37b1be0e721 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:15.985941 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002d1f3-9222-4676-9b78-c37b1be0e721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:15.988158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 99/183] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:15.988158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3f41b-144c-4243-ba8b-c18e7680d3ad None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:16.001124 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-5002d1f3-9222-4676-9b78-c37b1be0e721 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:16.002292 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5002d1f3-9222-4676-9b78-c37b1be0e721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:16.044936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:16.068548 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 85/184] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:16.069400 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002d1f3-9222-4676-9b78-c37b1be0e721 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:16.093346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d3f41b-144c-4243-ba8b-c18e7680d3ad None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:16.120791 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:16.128495 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97fe5f-8e95-475a-988b-d87b120d23c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:16.129376 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97fe5f-8e95-475a-988b-d87b120d23c4 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:16.129665 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97fe5f-8e95-475a-988b-d87b120d23c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:16.129935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97fe5f-8e95-475a-988b-d87b120d23c4 None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:16.131151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97fe5f-8e95-475a-988b-d87b120d23c4 None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:16.132603 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97fe5f-8e95-475a-988b-d87b120d23c4 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:16.134464 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 100/185] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:15 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:16.135468 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e1332-635a-4ec2-b991-99279ddc51ae None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:16.135756 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e1332-635a-4ec2-b991-99279ddc51ae None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:16.136035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e1332-635a-4ec2-b991-99279ddc51ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:16.151758 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-279e1332-635a-4ec2-b991-99279ddc51ae None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:16.152935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-279e1332-635a-4ec2-b991-99279ddc51ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:16.173595 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002d1f3-9222-4676-9b78-c37b1be0e721 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:16.206363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232c882-a6e9-48af-a0ea-d0ac78e72deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:16.207260 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232c882-a6e9-48af-a0ea-d0ac78e72deb None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:16.207544 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232c882-a6e9-48af-a0ea-d0ac78e72deb None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:16.207829 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232c882-a6e9-48af-a0ea-d0ac78e72deb None admin] PATH_INFO: `/v3/projects/d8f593f0a0934e5f9f2baf99016b98a7` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:16.208987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232c882-a6e9-48af-a0ea-d0ac78e72deb None admin] RBAC: Authorizing `identity:get_project(project_id=d8f593f0a0934e5f9f2baf99016b98a7)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:16.210440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e232c882-a6e9-48af-a0ea-d0ac78e72deb None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:16.212252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 86/186] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:16 2022] GET /identity/v3/projects/d8f593f0a0934e5f9f2baf99016b98a7 => generated 312 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:16.213018 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232c882-a6e9-48af-a0ea-d0ac78e72deb None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:16.220661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 101/187] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:16.221497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e1332-635a-4ec2-b991-99279ddc51ae None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:16.280209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:16.319674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232c882-a6e9-48af-a0ea-d0ac78e72deb None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:16.323338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e1332-635a-4ec2-b991-99279ddc51ae None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:16.338892 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:16.354009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471c31d-b5d1-4875-9276-f24d345a11e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:16.354841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471c31d-b5d1-4875-9276-f24d345a11e0 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:16.355155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471c31d-b5d1-4875-9276-f24d345a11e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:16.355433 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471c31d-b5d1-4875-9276-f24d345a11e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:16.355977 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7569b67-5180-44f5-9702-9087d1b37bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:16.356639 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7569b67-5180-44f5-9702-9087d1b37bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:16.356920 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7569b67-5180-44f5-9702-9087d1b37bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:16.357196 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7569b67-5180-44f5-9702-9087d1b37bf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:16.358024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7569b67-5180-44f5-9702-9087d1b37bf4 None admin] RBAC: Adding query filter params (role.id=b59c5b350c854281838a22884fd8f32f, scope.project.id=db74f3f0803a45359025fb277815314b, user.id=c23f5930d01c4d6f819cc9308da0b14e) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:16.358334 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7569b67-5180-44f5-9702-9087d1b37bf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:16.359820 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7569b67-5180-44f5-9702-9087d1b37bf4 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:16.379369 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 102/188] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:16 2022] GET /identity/v3/role_assignments?user.id=c23f5930d01c4d6f819cc9308da0b14e&scope.project.id=db74f3f0803a45359025fb277815314b&role.id=b59c5b350c854281838a22884fd8f32f => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:16.379566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471c31d-b5d1-4875-9276-f24d345a11e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:16.379648 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7569b67-5180-44f5-9702-9087d1b37bf4 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:16.379742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-4471c31d-b5d1-4875-9276-f24d345a11e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:16.379742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:16.379742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:16.379742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:16.379742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:16.379742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:16.379742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:16.379742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:16.379742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:16.379742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:16.379742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:16.379742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:16.379742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:16.379742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:16.379742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:16.379742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:16.380584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:16.380584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:16.380584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:16.380584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:16.380584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:16.380584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:16.380584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:16.380584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:16.380584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:16.380584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:16.380584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:16.380584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:16.380584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:16.380584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:16.380584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:16.380584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:16.381382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:16.381382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:16.381382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:16.381382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:16.381382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:16.381382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:16.381382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:16.381382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:16.381382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:16.381382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:16.381382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:16.381382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:16.381382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 87/189] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:30:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:16.381382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471c31d-b5d1-4875-9276-f24d345a11e0 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:16.476755 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7569b67-5180-44f5-9702-9087d1b37bf4 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:16.497439 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:16.505551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471c31d-b5d1-4875-9276-f24d345a11e0 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:16.520958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e6aaa-5def-4b73-a6e5-d2ac9d562fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:16.521871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e6aaa-5def-4b73-a6e5-d2ac9d562fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:16.522154 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e6aaa-5def-4b73-a6e5-d2ac9d562fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:16.522427 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e6aaa-5def-4b73-a6e5-d2ac9d562fe0 None admin] PATH_INFO: `/v3/users/ae13961e0d9143a99555aa604e417c34` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:16.537397 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e6aaa-5def-4b73-a6e5-d2ac9d562fe0 None admin] RBAC: Authorizing `identity:get_user(user_id=ae13961e0d9143a99555aa604e417c34)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:16.539230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e6aaa-5def-4b73-a6e5-d2ac9d562fe0 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:16.541049 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 103/190] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:16 2022] GET /identity/v3/users/ae13961e0d9143a99555aa604e417c34 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:16.541824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e6aaa-5def-4b73-a6e5-d2ac9d562fe0 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:16.546290 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e93f2-f66a-4e6a-8ae5-2fe6d4596594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:16.547374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e93f2-f66a-4e6a-8ae5-2fe6d4596594 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:16.547740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e93f2-f66a-4e6a-8ae5-2fe6d4596594 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:16.548019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e93f2-f66a-4e6a-8ae5-2fe6d4596594 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:16.560247 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e93f2-f66a-4e6a-8ae5-2fe6d4596594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:16.569112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-bd1e93f2-f66a-4e6a-8ae5-2fe6d4596594 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:16.569112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:16.569112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:16.569112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:16.569112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:16.569112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:16.569112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:16.569112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:16.569112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:16.569112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:16.569112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:16.569112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:16.569112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:16.569112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:16.569112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:16.569112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:16.570048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:16.570048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:16.570048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:16.570048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:16.570048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:16.570048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:16.570048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:16.570048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:16.570048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:16.570048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:16.570048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:16.570048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:16.570048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:16.570048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:16.570048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:16.570048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:16.570834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:16.570834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:16.570834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:16.570834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:16.570834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:16.570834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:16.570834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:16.570834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:16.570834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:16.570834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:16.570834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:16.570834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:16.570834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 88/191] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:16.574064 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e93f2-f66a-4e6a-8ae5-2fe6d4596594 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:16.624040 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:16.655790 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e6aaa-5def-4b73-a6e5-d2ac9d562fe0 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:16.684950 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e93f2-f66a-4e6a-8ae5-2fe6d4596594 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:16.689457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6a817-7b87-4d56-96a9-01515915e9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:16.690130 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6a817-7b87-4d56-96a9-01515915e9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:16.690411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6a817-7b87-4d56-96a9-01515915e9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:16.690690 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6a817-7b87-4d56-96a9-01515915e9d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:16.691587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6a817-7b87-4d56-96a9-01515915e9d7 None admin] RBAC: Adding query filter params (role.id=a43ba93f358b4b1a8a445851a5a216a0, scope.project.id=db74f3f0803a45359025fb277815314b, user.id=c23f5930d01c4d6f819cc9308da0b14e) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:16.691900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6a817-7b87-4d56-96a9-01515915e9d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:16.693363 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6a817-7b87-4d56-96a9-01515915e9d7 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:16.699136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 104/192] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:16 2022] GET /identity/v3/role_assignments?user.id=c23f5930d01c4d6f819cc9308da0b14e&scope.project.id=db74f3f0803a45359025fb277815314b&role.id=a43ba93f358b4b1a8a445851a5a216a0 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:16.699921 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6a817-7b87-4d56-96a9-01515915e9d7 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:16.724059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae549d00-2e87-4f8a-88fe-6e9043739cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:16.724861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae549d00-2e87-4f8a-88fe-6e9043739cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:16.725161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae549d00-2e87-4f8a-88fe-6e9043739cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:16.725484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae549d00-2e87-4f8a-88fe-6e9043739cb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:16.726359 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae549d00-2e87-4f8a-88fe-6e9043739cb1 None admin] RBAC: Adding query filter params (role.id=b59c5b350c854281838a22884fd8f32f, scope.project.id=d8f593f0a0934e5f9f2baf99016b98a7, user.id=c23f5930d01c4d6f819cc9308da0b14e) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:16.726693 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae549d00-2e87-4f8a-88fe-6e9043739cb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:16.728301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae549d00-2e87-4f8a-88fe-6e9043739cb1 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:16.735617 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 89/193] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:16 2022] GET /identity/v3/role_assignments?user.id=c23f5930d01c4d6f819cc9308da0b14e&scope.project.id=d8f593f0a0934e5f9f2baf99016b98a7&role.id=b59c5b350c854281838a22884fd8f32f => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:16.736678 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae549d00-2e87-4f8a-88fe-6e9043739cb1 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:16.770884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:16.859441 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6a817-7b87-4d56-96a9-01515915e9d7 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:16.880490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae549d00-2e87-4f8a-88fe-6e9043739cb1 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:16.885177 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:16.918791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a37f6-600d-4baa-8f09-df9fbd20bbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:16.919633 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a37f6-600d-4baa-8f09-df9fbd20bbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:16.919925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a37f6-600d-4baa-8f09-df9fbd20bbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:16.920195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a37f6-600d-4baa-8f09-df9fbd20bbe8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:16.922080 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a37f6-600d-4baa-8f09-df9fbd20bbe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:16.923604 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a37f6-600d-4baa-8f09-df9fbd20bbe8 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:16.925570 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 105/194] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:30:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:16.926359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a37f6-600d-4baa-8f09-df9fbd20bbe8 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:16.930480 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f722468d-fb59-4295-95fa-d8348429cb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:16.932626 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722468d-fb59-4295-95fa-d8348429cb3f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:16.932909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722468d-fb59-4295-95fa-d8348429cb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:16.933188 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722468d-fb59-4295-95fa-d8348429cb3f None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:16.933953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f722468d-fb59-4295-95fa-d8348429cb3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:16.934258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f722468d-fb59-4295-95fa-d8348429cb3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:16.936629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f722468d-fb59-4295-95fa-d8348429cb3f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:16.946224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 90/195] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:30:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:16.947107 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f722468d-fb59-4295-95fa-d8348429cb3f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:17.066118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a37f6-600d-4baa-8f09-df9fbd20bbe8 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:17.077984 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:17.083979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f722468d-fb59-4295-95fa-d8348429cb3f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:17.103199 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178b520-7c4c-4902-9c10-2ecbf75cabfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:17.103852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178b520-7c4c-4902-9c10-2ecbf75cabfc None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:17.104130 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178b520-7c4c-4902-9c10-2ecbf75cabfc None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:17.104399 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178b520-7c4c-4902-9c10-2ecbf75cabfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:17.122372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8e00f-685f-4c3a-931c-a5730854b66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:17.122965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8e00f-685f-4c3a-931c-a5730854b66f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:17.123280 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8e00f-685f-4c3a-931c-a5730854b66f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:17.123549 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8e00f-685f-4c3a-931c-a5730854b66f None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:17.124300 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8e00f-685f-4c3a-931c-a5730854b66f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:17.124602 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8e00f-685f-4c3a-931c-a5730854b66f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:17.125995 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8e00f-685f-4c3a-931c-a5730854b66f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:17.134160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 91/196] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:17.139067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8e00f-685f-4c3a-931c-a5730854b66f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:17.165451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:17.173228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178b520-7c4c-4902-9c10-2ecbf75cabfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:17.174706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178b520-7c4c-4902-9c10-2ecbf75cabfc None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:17.236310 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8e00f-685f-4c3a-931c-a5730854b66f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:17.271588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6697ad5-91b8-49fb-957b-7c63a6421fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:17.272272 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6697ad5-91b8-49fb-957b-7c63a6421fbe None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:17.272555 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6697ad5-91b8-49fb-957b-7c63a6421fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:17.272829 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6697ad5-91b8-49fb-957b-7c63a6421fbe None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:17.273547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6697ad5-91b8-49fb-957b-7c63a6421fbe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:17.274064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 106/197] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:17.275078 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178b520-7c4c-4902-9c10-2ecbf75cabfc None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:17.275190 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6697ad5-91b8-49fb-957b-7c63a6421fbe None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:17.285831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:17.307046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 92/198] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:30:17 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:30:17.307046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6697ad5-91b8-49fb-957b-7c63a6421fbe None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:17.395697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178b520-7c4c-4902-9c10-2ecbf75cabfc None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:17.402824 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:17.422425 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6697ad5-91b8-49fb-957b-7c63a6421fbe None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:17.441898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256856b8-1aef-4819-ac51-69186725b0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:17.442804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256856b8-1aef-4819-ac51-69186725b0ad None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:17.443114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256856b8-1aef-4819-ac51-69186725b0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:17.443485 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256856b8-1aef-4819-ac51-69186725b0ad None admin] PATH_INFO: `/v3/users/ae13961e0d9143a99555aa604e417c34` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:17.445990 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256856b8-1aef-4819-ac51-69186725b0ad None admin] RBAC: Authorizing `identity:get_user(user_id=ae13961e0d9143a99555aa604e417c34)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:17.448162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256856b8-1aef-4819-ac51-69186725b0ad None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:17.451047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 107/199] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:17 2022] GET /identity/v3/users/ae13961e0d9143a99555aa604e417c34 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:17.451879 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256856b8-1aef-4819-ac51-69186725b0ad None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:17.473677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d9edd-faf4-4f15-b2b3-13784805d026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:17.474714 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d9edd-faf4-4f15-b2b3-13784805d026 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:17.475107 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d9edd-faf4-4f15-b2b3-13784805d026 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:17.475452 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d9edd-faf4-4f15-b2b3-13784805d026 None admin] PATH_INFO: `/v3/users/aa86fe483ab441bcbb42b09b4c81accc` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:17.476776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d9edd-faf4-4f15-b2b3-13784805d026 None admin] RBAC: Authorizing `identity:get_user(user_id=aa86fe483ab441bcbb42b09b4c81accc)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:17.478293 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d9edd-faf4-4f15-b2b3-13784805d026 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:17.481084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 93/200] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:17 2022] GET /identity/v3/users/aa86fe483ab441bcbb42b09b4c81accc => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:17.481084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d9edd-faf4-4f15-b2b3-13784805d026 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:17.555913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:17.594519 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256856b8-1aef-4819-ac51-69186725b0ad None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:17.619875 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d9edd-faf4-4f15-b2b3-13784805d026 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:17.639992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2dca2-a72e-49f6-9f4e-0db401737c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:17.640885 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2dca2-a72e-49f6-9f4e-0db401737c29 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:17.641164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2dca2-a72e-49f6-9f4e-0db401737c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:17.641439 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2dca2-a72e-49f6-9f4e-0db401737c29 None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:17.645179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2dca2-a72e-49f6-9f4e-0db401737c29 None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:17.647149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2dca2-a72e-49f6-9f4e-0db401737c29 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:17.648736 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 108/201] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:17 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:17.651094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2dca2-a72e-49f6-9f4e-0db401737c29 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:17.666226 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9a87c-c82e-46e4-99ba-5eb9f78c2385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:17.666584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9a87c-c82e-46e4-99ba-5eb9f78c2385 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:17.668014 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9a87c-c82e-46e4-99ba-5eb9f78c2385 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:17.668294 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9a87c-c82e-46e4-99ba-5eb9f78c2385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:17.702982 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:17.761682 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9a87c-c82e-46e4-99ba-5eb9f78c2385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:17.763613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9a87c-c82e-46e4-99ba-5eb9f78c2385 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:17.776047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2dca2-a72e-49f6-9f4e-0db401737c29 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:17.830641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c34fe-2656-4e57-8d8e-5ceee9cd6af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:17.831822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c34fe-2656-4e57-8d8e-5ceee9cd6af8 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:17.832342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c34fe-2656-4e57-8d8e-5ceee9cd6af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:17.833075 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c34fe-2656-4e57-8d8e-5ceee9cd6af8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:17.855069 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:17.899948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 94/202] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:17.901025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9a87c-c82e-46e4-99ba-5eb9f78c2385 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:17.936263 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c34fe-2656-4e57-8d8e-5ceee9cd6af8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:17.938991 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c34fe-2656-4e57-8d8e-5ceee9cd6af8 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:17.961426 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:18.027255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9a87c-c82e-46e4-99ba-5eb9f78c2385 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:18.075532 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 109/203] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:18.075532 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c34fe-2656-4e57-8d8e-5ceee9cd6af8 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:18.084509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed17fdf-e38b-405a-871b-790e52bf356d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:18.085814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed17fdf-e38b-405a-871b-790e52bf356d None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:18.086560 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed17fdf-e38b-405a-871b-790e52bf356d None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:18.087194 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed17fdf-e38b-405a-871b-790e52bf356d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:18.088499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed17fdf-e38b-405a-871b-790e52bf356d None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=8855f4be124f44818348e5b04172899b, user.id=ae13961e0d9143a99555aa604e417c34) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:18.089421 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed17fdf-e38b-405a-871b-790e52bf356d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:18.091624 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed17fdf-e38b-405a-871b-790e52bf356d None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:18.100126 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 95/204] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:17 2022] GET /identity/v3/role_assignments?user.id=ae13961e0d9143a99555aa604e417c34&scope.project.id=8855f4be124f44818348e5b04172899b&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:18.101839 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed17fdf-e38b-405a-871b-790e52bf356d None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:18.154518 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:18.230480 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed17fdf-e38b-405a-871b-790e52bf356d None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:18.252703 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:18.283493 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f81029-d5ef-4218-8ee1-103a4826bfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:18.284264 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f81029-d5ef-4218-8ee1-103a4826bfed None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:18.284550 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f81029-d5ef-4218-8ee1-103a4826bfed None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:18.284826 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f81029-d5ef-4218-8ee1-103a4826bfed None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:18.287092 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f81029-d5ef-4218-8ee1-103a4826bfed None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:18.289066 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f81029-d5ef-4218-8ee1-103a4826bfed None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:18.291115 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 96/205] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:18 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:18.300862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f81029-d5ef-4218-8ee1-103a4826bfed None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:18.306884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c34fe-2656-4e57-8d8e-5ceee9cd6af8 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:18.364297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79ed22-bb5b-4780-bffb-0bc5024dae0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:18.366500 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79ed22-bb5b-4780-bffb-0bc5024dae0b None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:18.369154 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79ed22-bb5b-4780-bffb-0bc5024dae0b None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:18.369668 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79ed22-bb5b-4780-bffb-0bc5024dae0b None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:18.371136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79ed22-bb5b-4780-bffb-0bc5024dae0b None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:18.373621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79ed22-bb5b-4780-bffb-0bc5024dae0b None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:18.380791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 110/206] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:18 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:18.386624 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79ed22-bb5b-4780-bffb-0bc5024dae0b None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:18.509230 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f81029-d5ef-4218-8ee1-103a4826bfed None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:18.582317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79ed22-bb5b-4780-bffb-0bc5024dae0b None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:18.585177 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3daaf-e946-4091-9971-e77b686ea322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:18.587052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3daaf-e946-4091-9971-e77b686ea322 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:18.587052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3daaf-e946-4091-9971-e77b686ea322 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:18.587052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3daaf-e946-4091-9971-e77b686ea322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:18.587435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3daaf-e946-4091-9971-e77b686ea322 None admin] RBAC: Adding query filter params (role.id=784d41de5e1d415b93a1d4e0dc5c237b, scope.project.id=db74f3f0803a45359025fb277815314b, user.id=aa86fe483ab441bcbb42b09b4c81accc) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:18.587752 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3daaf-e946-4091-9971-e77b686ea322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:18.589224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3daaf-e946-4091-9971-e77b686ea322 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:18.602748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 97/207] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:18 2022] GET /identity/v3/role_assignments?user.id=aa86fe483ab441bcbb42b09b4c81accc&scope.project.id=db74f3f0803a45359025fb277815314b&role.id=784d41de5e1d415b93a1d4e0dc5c237b => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:18.610057 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:18.642529 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51fc55d-f682-4786-b2ef-f890761a85f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:18.643301 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51fc55d-f682-4786-b2ef-f890761a85f0 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:18.643589 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51fc55d-f682-4786-b2ef-f890761a85f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:18.643868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51fc55d-f682-4786-b2ef-f890761a85f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:18.644755 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51fc55d-f682-4786-b2ef-f890761a85f0 None admin] RBAC: Adding query filter params (role.id=a43ba93f358b4b1a8a445851a5a216a0, scope.project.id=8855f4be124f44818348e5b04172899b, user.id=ae13961e0d9143a99555aa604e417c34) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:18.645081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51fc55d-f682-4786-b2ef-f890761a85f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:18.646602 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51fc55d-f682-4786-b2ef-f890761a85f0 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:18.653598 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 111/208] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:18 2022] GET /identity/v3/role_assignments?user.id=ae13961e0d9143a99555aa604e417c34&scope.project.id=8855f4be124f44818348e5b04172899b&role.id=a43ba93f358b4b1a8a445851a5a216a0 => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:19.660874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15803e0e-15ab-498d-ad15-6765123956c6 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:19.661428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15803e0e-15ab-498d-ad15-6765123956c6 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:19.661428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15803e0e-15ab-498d-ad15-6765123956c6 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:19.665657 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 98/209] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:19.671179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abceb561-14ed-4a4e-ac26-362293f557cc None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:19.671979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abceb561-14ed-4a4e-ac26-362293f557cc None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:19.672424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abceb561-14ed-4a4e-ac26-362293f557cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:19.675485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345483a8-0fa3-43cd-a174-5e507fca2224 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:19.675987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345483a8-0fa3-43cd-a174-5e507fca2224 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:19.676482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345483a8-0fa3-43cd-a174-5e507fca2224 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:19.678017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 99/210] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:30:19.683436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae62f38-1c95-4a22-b8fb-4ead3c7dba0a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:19.684087 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae62f38-1c95-4a22-b8fb-4ead3c7dba0a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:19.684550 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae62f38-1c95-4a22-b8fb-4ead3c7dba0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:19.692633 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-abceb561-14ed-4a4e-ac26-362293f557cc None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:19.693898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-abceb561-14ed-4a4e-ac26-362293f557cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:19.705819 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-cae62f38-1c95-4a22-b8fb-4ead3c7dba0a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:19.707231 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-cae62f38-1c95-4a22-b8fb-4ead3c7dba0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:19.765614 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:19.773802 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 112/211] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:19.796466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 100/212] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:19.801153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abceb561-14ed-4a4e-ac26-362293f557cc None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:19.822533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae62f38-1c95-4a22-b8fb-4ead3c7dba0a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:19.927260 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abceb561-14ed-4a4e-ac26-362293f557cc None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:19.936421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:19.957631 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae62f38-1c95-4a22-b8fb-4ead3c7dba0a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:19.964147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43171b8-3dfc-4baa-a7d3-3a49a2d18bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:19.965146 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43171b8-3dfc-4baa-a7d3-3a49a2d18bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:19.965568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43171b8-3dfc-4baa-a7d3-3a49a2d18bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:19.965976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43171b8-3dfc-4baa-a7d3-3a49a2d18bb6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:19.967322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43171b8-3dfc-4baa-a7d3-3a49a2d18bb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:19.968950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43171b8-3dfc-4baa-a7d3-3a49a2d18bb6 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:19.970731 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 113/213] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:30:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:19.976098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43171b8-3dfc-4baa-a7d3-3a49a2d18bb6 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:20.001862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b337bcc-133a-4a07-99d3-39e47b9866f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:20.003003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b337bcc-133a-4a07-99d3-39e47b9866f5 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:20.003571 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b337bcc-133a-4a07-99d3-39e47b9866f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:20.004122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b337bcc-133a-4a07-99d3-39e47b9866f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:20.015166 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b337bcc-133a-4a07-99d3-39e47b9866f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:20.024628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-9b337bcc-133a-4a07-99d3-39e47b9866f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:20.024628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:20.024628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:20.024628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:20.024628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:20.024628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:20.024628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:20.024628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:20.024628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:20.024628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:20.024628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:20.024628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:20.024628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:20.024628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:20.024628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:20.024628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:20.025557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:20.025557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:20.025557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:20.025557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:20.025557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:20.025557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:20.025557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:20.025557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:20.025557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:20.025557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:20.025557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:20.025557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:20.025557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:20.025557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:20.025557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:20.025557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:20.026336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:20.026336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:20.026336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:20.026336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:20.026336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:20.026336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:20.026336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:20.026336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:20.026336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:20.026336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:20.026336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:20.026336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:20.027411 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 101/214] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:30:20.031946 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b337bcc-133a-4a07-99d3-39e47b9866f5 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:20.084873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:20.094246 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43171b8-3dfc-4baa-a7d3-3a49a2d18bb6 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:20.135341 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf74f6-bbd3-43c4-94a3-7011da8e232f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:20.138040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf74f6-bbd3-43c4-94a3-7011da8e232f None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:20.139208 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf74f6-bbd3-43c4-94a3-7011da8e232f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:20.139747 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf74f6-bbd3-43c4-94a3-7011da8e232f None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:20.143164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf74f6-bbd3-43c4-94a3-7011da8e232f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:20.146943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf74f6-bbd3-43c4-94a3-7011da8e232f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:20.150961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b337bcc-133a-4a07-99d3-39e47b9866f5 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:20.179628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 114/215] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:30:19 2022] POST /identity/v3/users => generated 297 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:30:20.211907 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76712fb2-dcdf-4360-8c6f-35aba838f5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:20.213038 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76712fb2-dcdf-4360-8c6f-35aba838f5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:20.213540 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76712fb2-dcdf-4360-8c6f-35aba838f5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:20.214109 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76712fb2-dcdf-4360-8c6f-35aba838f5b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:20.215249 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76712fb2-dcdf-4360-8c6f-35aba838f5b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:20.215797 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76712fb2-dcdf-4360-8c6f-35aba838f5b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:20.217583 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76712fb2-dcdf-4360-8c6f-35aba838f5b8 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:20.228469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 102/216] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:20.235514 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf74f6-bbd3-43c4-94a3-7011da8e232f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:20.271575 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:20.334896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c38bdd-2919-4e17-80d9-62e3987326aa None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:20.335274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c38bdd-2919-4e17-80d9-62e3987326aa None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:20.335497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c38bdd-2919-4e17-80d9-62e3987326aa None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:20.337164 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 103/217] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:20.342986 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542c1e5-1208-441a-8c6a-da3b6df980a0 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:20.343584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542c1e5-1208-441a-8c6a-da3b6df980a0 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:20.344075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542c1e5-1208-441a-8c6a-da3b6df980a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:20.354914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf74f6-bbd3-43c4-94a3-7011da8e232f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:20.363509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-2542c1e5-1208-441a-8c6a-da3b6df980a0 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:20.364984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2542c1e5-1208-441a-8c6a-da3b6df980a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:20.405085 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f0785-0874-4de9-acbf-70f6ec6b5b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:20.406047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f0785-0874-4de9-acbf-70f6ec6b5b05 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:20.406258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f0785-0874-4de9-acbf-70f6ec6b5b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:20.406534 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f0785-0874-4de9-acbf-70f6ec6b5b05 None admin] PATH_INFO: `/v3/users/a5383a4dbe534b748e4240c0298eb60e` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:20.425872 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256f0785-0874-4de9-acbf-70f6ec6b5b05 None admin] RBAC: Authorizing `identity:get_user(user_id=a5383a4dbe534b748e4240c0298eb60e)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:20.427451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256f0785-0874-4de9-acbf-70f6ec6b5b05 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:20.430855 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 115/218] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:20 2022] GET /identity/v3/users/a5383a4dbe534b748e4240c0298eb60e => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:20.436465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f0785-0874-4de9-acbf-70f6ec6b5b05 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:20.453547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 104/219] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:20.476374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542c1e5-1208-441a-8c6a-da3b6df980a0 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:20.522734 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:20.566277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:20.586169 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f0785-0874-4de9-acbf-70f6ec6b5b05 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:20.624844 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2542c1e5-1208-441a-8c6a-da3b6df980a0 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:20.627823 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a533c-3400-454c-9131-1154778371d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:20.628784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a533c-3400-454c-9131-1154778371d1 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:20.629274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a533c-3400-454c-9131-1154778371d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:20.629774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a533c-3400-454c-9131-1154778371d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:20.678754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781f96a-1317-4e61-a0ce-d82b3d612b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:20.680193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781f96a-1317-4e61-a0ce-d82b3d612b50 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:20.680661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781f96a-1317-4e61-a0ce-d82b3d612b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:20.681872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781f96a-1317-4e61-a0ce-d82b3d612b50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:20.697980 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781f96a-1317-4e61-a0ce-d82b3d612b50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:20.708026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-e781f96a-1317-4e61-a0ce-d82b3d612b50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:20.708026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:20.708026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:20.708026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:20.708026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:20.708026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:20.708026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:20.708026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:20.708026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:20.708026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:20.708026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:20.708026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:20.708026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:20.708026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:20.708026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:20.708026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:20.709122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:20.709122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:20.709122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:20.709122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:20.709122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:20.709122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:20.709122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:20.709122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:20.709122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:20.709122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:20.709122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:20.709122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:20.709122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:20.709122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:20.709122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:20.709122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:20.710099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:20.710099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:20.710099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:20.710099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:20.710099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:20.710099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:20.710099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:20.710099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:20.710099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:20.710099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:20.710099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:20.710099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:20.712987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 105/220] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:30:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:20.720197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781f96a-1317-4e61-a0ce-d82b3d612b50 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:20.735418 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a533c-3400-454c-9131-1154778371d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:20.737710 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a533c-3400-454c-9131-1154778371d1 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:20.833600 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:20.848969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 116/221] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:20.850775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ee674-9968-4ca7-a966-2c5cb1e6d922 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:20.851295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ee674-9968-4ca7-a966-2c5cb1e6d922 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:20.851791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ee674-9968-4ca7-a966-2c5cb1e6d922 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:20.853640 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 117/222] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:20.854704 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ee674-9968-4ca7-a966-2c5cb1e6d922 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:20.879784 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:20.902078 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781f96a-1317-4e61-a0ce-d82b3d612b50 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:20.943563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe7b13-0ff0-41e2-aeaa-edf7cfaf1d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:20.944477 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe7b13-0ff0-41e2-aeaa-edf7cfaf1d5f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:20.944918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe7b13-0ff0-41e2-aeaa-edf7cfaf1d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:20.945334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe7b13-0ff0-41e2-aeaa-edf7cfaf1d5f None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:20.946421 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe7b13-0ff0-41e2-aeaa-edf7cfaf1d5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:20.946962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe7b13-0ff0-41e2-aeaa-edf7cfaf1d5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:20.948520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe7b13-0ff0-41e2-aeaa-edf7cfaf1d5f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:20.957996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 106/223] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:20.959603 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb0cab-3146-4e41-96bd-b301d44482ed None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:20.960021 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb0cab-3146-4e41-96bd-b301d44482ed None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:20.960418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb0cab-3146-4e41-96bd-b301d44482ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:20.968529 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ee674-9968-4ca7-a966-2c5cb1e6d922 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:20.977682 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-3cdb0cab-3146-4e41-96bd-b301d44482ed None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:20.978968 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-3cdb0cab-3146-4e41-96bd-b301d44482ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:21.005369 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c461a24-e71f-4c18-afdc-8e6e6b5f3d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:21.006595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c461a24-e71f-4c18-afdc-8e6e6b5f3d61 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:21.007109 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c461a24-e71f-4c18-afdc-8e6e6b5f3d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:21.007608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c461a24-e71f-4c18-afdc-8e6e6b5f3d61 None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:21.009074 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c461a24-e71f-4c18-afdc-8e6e6b5f3d61 None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:21.010827 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c461a24-e71f-4c18-afdc-8e6e6b5f3d61 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:21.012931 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 118/224] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:20 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:21.014074 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c461a24-e71f-4c18-afdc-8e6e6b5f3d61 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:21.063086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 107/225] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:21.064330 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb0cab-3146-4e41-96bd-b301d44482ed None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:21.083379 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:21.120194 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c461a24-e71f-4c18-afdc-8e6e6b5f3d61 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:21.155521 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8af98b-bcb8-49a9-971e-0f9af28e122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:21.156771 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8af98b-bcb8-49a9-971e-0f9af28e122e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:21.157282 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8af98b-bcb8-49a9-971e-0f9af28e122e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:21.157755 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8af98b-bcb8-49a9-971e-0f9af28e122e None admin] PATH_INFO: `/v3/users/ae13961e0d9143a99555aa604e417c34` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:21.159275 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8af98b-bcb8-49a9-971e-0f9af28e122e None admin] RBAC: Authorizing `identity:get_user(user_id=ae13961e0d9143a99555aa604e417c34)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:21.160987 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8af98b-bcb8-49a9-971e-0f9af28e122e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:21.163269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 119/226] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:21 2022] GET /identity/v3/users/ae13961e0d9143a99555aa604e417c34 => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:21.168907 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8af98b-bcb8-49a9-971e-0f9af28e122e None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:21.169309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:21.190224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb0cab-3146-4e41-96bd-b301d44482ed None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:21.228895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84fea4-709d-4581-bd29-6c865ffb669c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:21.229367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84fea4-709d-4581-bd29-6c865ffb669c None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:21.229656 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84fea4-709d-4581-bd29-6c865ffb669c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:21.229936 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84fea4-709d-4581-bd29-6c865ffb669c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:21.230791 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84fea4-709d-4581-bd29-6c865ffb669c None admin] RBAC: Adding query filter params (role.id=b59c5b350c854281838a22884fd8f32f, scope.project.id=8855f4be124f44818348e5b04172899b, user.id=a5383a4dbe534b748e4240c0298eb60e) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:21.231116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84fea4-709d-4581-bd29-6c865ffb669c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:21.232688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84fea4-709d-4581-bd29-6c865ffb669c None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:21.238977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 108/227] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:21 2022] GET /identity/v3/role_assignments?user.id=a5383a4dbe534b748e4240c0298eb60e&scope.project.id=8855f4be124f44818348e5b04172899b&role.id=b59c5b350c854281838a22884fd8f32f => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:21.239816 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84fea4-709d-4581-bd29-6c865ffb669c None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:21.277924 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8af98b-bcb8-49a9-971e-0f9af28e122e None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:21.304527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:21.321661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364e43a-14e7-4d51-a354-2883e1395321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:21.322933 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364e43a-14e7-4d51-a354-2883e1395321 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:21.323984 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364e43a-14e7-4d51-a354-2883e1395321 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:21.325371 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364e43a-14e7-4d51-a354-2883e1395321 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:21.351172 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364e43a-14e7-4d51-a354-2883e1395321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:21.363183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-e364e43a-14e7-4d51-a354-2883e1395321 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:21.363183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:21.363183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:21.363183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:21.363183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:21.363183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:21.363183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:21.363183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:21.363183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:21.363183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:21.363183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:21.363183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:21.363183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:21.363183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:21.363183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:21.363183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:21.364207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:21.364207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:21.364207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:21.364207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:21.364207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:21.364207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:21.364207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:21.364207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:21.364207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:21.364207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:21.364207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:21.364207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:21.364207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:21.364207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:21.364207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:21.364207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:21.365007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:21.365007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:21.365007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:21.365007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:21.365007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:21.365007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:21.365007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:21.365007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:21.365007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:21.365007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:21.365007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:21.365007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:21.368913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 120/228] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:30:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:21.377724 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364e43a-14e7-4d51-a354-2883e1395321 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:21.430167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84fea4-709d-4581-bd29-6c865ffb669c None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:21.497511 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f984f32-5c6a-4e41-82ec-38bbb6f33fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:21.498309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f984f32-5c6a-4e41-82ec-38bbb6f33fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:21.498614 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f984f32-5c6a-4e41-82ec-38bbb6f33fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:21.499077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f984f32-5c6a-4e41-82ec-38bbb6f33fb3 None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:21.500151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f984f32-5c6a-4e41-82ec-38bbb6f33fb3 None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:21.501609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f984f32-5c6a-4e41-82ec-38bbb6f33fb3 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:21.503386 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 109/229] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:21 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:21.510700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f984f32-5c6a-4e41-82ec-38bbb6f33fb3 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:21.516775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364e43a-14e7-4d51-a354-2883e1395321 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:21.532418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:21.590220 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99249106-ec64-4dfb-8576-c4b54d66cf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:21.590954 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99249106-ec64-4dfb-8576-c4b54d66cf79 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:21.591286 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99249106-ec64-4dfb-8576-c4b54d66cf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:21.591564 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99249106-ec64-4dfb-8576-c4b54d66cf79 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:21.592330 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99249106-ec64-4dfb-8576-c4b54d66cf79 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:21.592642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99249106-ec64-4dfb-8576-c4b54d66cf79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:21.594051 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99249106-ec64-4dfb-8576-c4b54d66cf79 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:21.609819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 121/230] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:30:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:21.615447 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99249106-ec64-4dfb-8576-c4b54d66cf79 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:21.703923 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f984f32-5c6a-4e41-82ec-38bbb6f33fb3 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:21.707170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:21.774221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db0f2e-1135-4b02-874a-ae4785325a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:21.775094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db0f2e-1135-4b02-874a-ae4785325a74 None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:21.775380 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db0f2e-1135-4b02-874a-ae4785325a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:21.775659 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db0f2e-1135-4b02-874a-ae4785325a74 None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b/users/ae13961e0d9143a99555aa604e417c34/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:21.777866 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db0f2e-1135-4b02-874a-ae4785325a74 None admin] RBAC: Authorizing `identity:create_grant(project_id=8855f4be124f44818348e5b04172899b, user_id=ae13961e0d9143a99555aa604e417c34, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:21.780335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db0f2e-1135-4b02-874a-ae4785325a74 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:21.814095 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99249106-ec64-4dfb-8576-c4b54d66cf79 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:21.843938 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 110/231] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:30:21 2022] PUT /identity/v3/projects/8855f4be124f44818348e5b04172899b/users/ae13961e0d9143a99555aa604e417c34/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:30:21.852600 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab700ed-3ad4-4cf3-9123-9502693c7722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:21.854338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab700ed-3ad4-4cf3-9123-9502693c7722 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:21.854855 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab700ed-3ad4-4cf3-9123-9502693c7722 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:21.855486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab700ed-3ad4-4cf3-9123-9502693c7722 None admin] PATH_INFO: `/v3/users/ae13961e0d9143a99555aa604e417c34` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:21.857113 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab700ed-3ad4-4cf3-9123-9502693c7722 None admin] RBAC: Authorizing `identity:get_user(user_id=ae13961e0d9143a99555aa604e417c34)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:21.858815 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab700ed-3ad4-4cf3-9123-9502693c7722 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:21.861089 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 122/232] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:21 2022] GET /identity/v3/users/ae13961e0d9143a99555aa604e417c34 => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:21.867093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db0f2e-1135-4b02-874a-ae4785325a74 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:22.075083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:22.101186 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db0f2e-1135-4b02-874a-ae4785325a74 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:22.137033 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3b66c-0166-4afa-860b-06e165bf099d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:22.138039 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3b66c-0166-4afa-860b-06e165bf099d None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:22.138335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3b66c-0166-4afa-860b-06e165bf099d None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:22.138614 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3b66c-0166-4afa-860b-06e165bf099d None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:22.139871 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3b66c-0166-4afa-860b-06e165bf099d None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:22.141351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3b66c-0166-4afa-860b-06e165bf099d None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:22.143506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 111/233] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:21 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:22.149049 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab700ed-3ad4-4cf3-9123-9502693c7722 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:22.385353 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab700ed-3ad4-4cf3-9123-9502693c7722 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:22.409839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:22.444917 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3389f7-cd0e-411b-9381-4ca33103b560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:22.445831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3389f7-cd0e-411b-9381-4ca33103b560 None admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:22.446181 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3389f7-cd0e-411b-9381-4ca33103b560 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:22.446463 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3389f7-cd0e-411b-9381-4ca33103b560 None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b/users/ae13961e0d9143a99555aa604e417c34/roles/a43ba93f358b4b1a8a445851a5a216a0` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:22.452942 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3389f7-cd0e-411b-9381-4ca33103b560 None admin] RBAC: Authorizing `identity:create_grant(project_id=8855f4be124f44818348e5b04172899b, user_id=ae13961e0d9143a99555aa604e417c34, role_id=a43ba93f358b4b1a8a445851a5a216a0)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:22.454447 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3389f7-cd0e-411b-9381-4ca33103b560 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:22.487781 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 123/234] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:30:22 2022] PUT /identity/v3/projects/8855f4be124f44818348e5b04172899b/users/ae13961e0d9143a99555aa604e417c34/roles/a43ba93f358b4b1a8a445851a5a216a0 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:30:22.538935 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb498d6b-037a-4ab9-80e6-33032ee874d6 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:22.539281 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb498d6b-037a-4ab9-80e6-33032ee874d6 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:22.539541 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb498d6b-037a-4ab9-80e6-33032ee874d6 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:22.542382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 112/235] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:22.550450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5adb68-7e51-4371-9a26-b5ae7898f692 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:22.550809 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5adb68-7e51-4371-9a26-b5ae7898f692 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:22.551043 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5adb68-7e51-4371-9a26-b5ae7898f692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:22.576179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-fe5adb68-7e51-4371-9a26-b5ae7898f692 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:22.577453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-fe5adb68-7e51-4371-9a26-b5ae7898f692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:22.698105 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 124/236] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:22.723821 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb498d6b-037a-4ab9-80e6-33032ee874d6 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:22.836785 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb498d6b-037a-4ab9-80e6-33032ee874d6 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:22.881993 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf36c57-9ee7-4ac9-a3e0-4986774361c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:22.882809 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf36c57-9ee7-4ac9-a3e0-4986774361c1 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:22.883120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf36c57-9ee7-4ac9-a3e0-4986774361c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:22.883400 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf36c57-9ee7-4ac9-a3e0-4986774361c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:22.897295 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf36c57-9ee7-4ac9-a3e0-4986774361c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:22.905744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-5bf36c57-9ee7-4ac9-a3e0-4986774361c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:22.905744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:22.905744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:22.905744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:22.905744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:22.905744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:22.905744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:22.905744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:22.905744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:22.905744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:22.905744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:22.905744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:22.905744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:22.905744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:22.905744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:22.905744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:22.906616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:22.906616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:22.906616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:22.906616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:22.906616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:22.906616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:22.906616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:22.906616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:22.906616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:22.906616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:22.906616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:22.906616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:22.906616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:22.906616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:22.906616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:22.906616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:22.907416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:22.907416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:22.907416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:22.907416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:22.907416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:22.907416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:22.907416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:22.907416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:22.907416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:22.907416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:22.907416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:22.907416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:22.913620 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 113/237] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:22.919035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5adb68-7e51-4371-9a26-b5ae7898f692 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:22.984960 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50264d-2a36-4a60-a35d-ecda62925aa5 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:22.985363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50264d-2a36-4a60-a35d-ecda62925aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:22.985519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50264d-2a36-4a60-a35d-ecda62925aa5 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:22.986883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 114/238] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:22.994602 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b31684-5ae4-4789-8815-dce52df62270 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:22.995089 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b31684-5ae4-4789-8815-dce52df62270 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:22.995581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b31684-5ae4-4789-8815-dce52df62270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:23.018353 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:23.046901 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-85b31684-5ae4-4789-8815-dce52df62270 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:23.048115 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-85b31684-5ae4-4789-8815-dce52df62270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:23.050621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5adb68-7e51-4371-9a26-b5ae7898f692 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:23.064461 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:23.114227 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd31fc8-8630-4812-b53d-00b5d0efc7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:23.114914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd31fc8-8630-4812-b53d-00b5d0efc7bb None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:23.115228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd31fc8-8630-4812-b53d-00b5d0efc7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:23.115501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd31fc8-8630-4812-b53d-00b5d0efc7bb None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:23.122081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd31fc8-8630-4812-b53d-00b5d0efc7bb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:23.127050 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd31fc8-8630-4812-b53d-00b5d0efc7bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:23.127050 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd31fc8-8630-4812-b53d-00b5d0efc7bb None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:23.140993 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 125/239] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:23.148193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd31fc8-8630-4812-b53d-00b5d0efc7bb None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:23.190307 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 115/240] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:23.227411 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b31684-5ae4-4789-8815-dce52df62270 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:23.270908 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd31fc8-8630-4812-b53d-00b5d0efc7bb None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:23.305136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2ba97-edb0-44c2-aa60-1a9bb755ec8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:23.306154 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2ba97-edb0-44c2-aa60-1a9bb755ec8f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:23.306435 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2ba97-edb0-44c2-aa60-1a9bb755ec8f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:23.306708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2ba97-edb0-44c2-aa60-1a9bb755ec8f None admin] PATH_INFO: `/v3/users/acfdecf5ccda4611b4e87060de5eb0a8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:23.331045 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2ba97-edb0-44c2-aa60-1a9bb755ec8f None admin] RBAC: Authorizing `identity:get_user(user_id=acfdecf5ccda4611b4e87060de5eb0a8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:23.331852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2ba97-edb0-44c2-aa60-1a9bb755ec8f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:23.333629 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 126/241] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:23 2022] GET /identity/v3/users/acfdecf5ccda4611b4e87060de5eb0a8 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:23.344591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2ba97-edb0-44c2-aa60-1a9bb755ec8f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:23.399837 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:23.449985 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b31684-5ae4-4789-8815-dce52df62270 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:23.494150 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14246896-1e2c-43bf-8302-3d8ce92302f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:23.494962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14246896-1e2c-43bf-8302-3d8ce92302f2 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:23.495424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14246896-1e2c-43bf-8302-3d8ce92302f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:23.495820 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14246896-1e2c-43bf-8302-3d8ce92302f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:23.496999 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14246896-1e2c-43bf-8302-3d8ce92302f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:23.498618 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14246896-1e2c-43bf-8302-3d8ce92302f2 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:23.500620 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 116/242] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:30:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:23.501145 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:23.506412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14246896-1e2c-43bf-8302-3d8ce92302f2 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:23.620211 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2ba97-edb0-44c2-aa60-1a9bb755ec8f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:23.635743 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14246896-1e2c-43bf-8302-3d8ce92302f2 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:23.696410 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ffc63-02da-4684-94df-1373a57f7f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:23.697083 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ffc63-02da-4684-94df-1373a57f7f0e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:23.697360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ffc63-02da-4684-94df-1373a57f7f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:23.697638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ffc63-02da-4684-94df-1373a57f7f0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:23.706252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183923d7-32b4-4eca-8c40-91d4dc07947e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:23.706877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183923d7-32b4-4eca-8c40-91d4dc07947e None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:23.707183 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183923d7-32b4-4eca-8c40-91d4dc07947e None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:23.707459 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183923d7-32b4-4eca-8c40-91d4dc07947e None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:23.708178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183923d7-32b4-4eca-8c40-91d4dc07947e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:23.709638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183923d7-32b4-4eca-8c40-91d4dc07947e None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:23.755896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 117/243] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:30:23 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:30:23.769552 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26f937-a82d-4e5d-9716-95f070c9953e None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:23.769867 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26f937-a82d-4e5d-9716-95f070c9953e None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:23.770115 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26f937-a82d-4e5d-9716-95f070c9953e None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:23.771744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 118/244] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:30:23.778275 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ef551-5da4-4d93-9c32-cd9c6ac8c39a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:23.778613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ef551-5da4-4d93-9c32-cd9c6ac8c39a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:23.778840 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ef551-5da4-4d93-9c32-cd9c6ac8c39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:23.800556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-be6ef551-5da4-4d93-9c32-cd9c6ac8c39a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:23.801170 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-be6ef551-5da4-4d93-9c32-cd9c6ac8c39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:23.833963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ffc63-02da-4684-94df-1373a57f7f0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:23.835897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ffc63-02da-4684-94df-1373a57f7f0e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:23.861813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:23.885973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 119/245] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:23.911416 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:23.919067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ef551-5da4-4d93-9c32-cd9c6ac8c39a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:24.026744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ef551-5da4-4d93-9c32-cd9c6ac8c39a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:24.066677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1ee84-8a27-420f-bb50-6e5ab4d5bc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:24.067836 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1ee84-8a27-420f-bb50-6e5ab4d5bc19 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:24.068113 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1ee84-8a27-420f-bb50-6e5ab4d5bc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:24.068383 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa1ee84-8a27-420f-bb50-6e5ab4d5bc19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:24.081992 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa1ee84-8a27-420f-bb50-6e5ab4d5bc19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:24.090412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-9aa1ee84-8a27-420f-bb50-6e5ab4d5bc19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:24.090412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:24.090412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:24.090412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:24.090412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:24.090412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:24.090412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:24.090412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:24.090412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:24.090412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:24.090412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:24.090412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:24.090412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:24.090412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:24.090412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:24.090412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:24.091419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:24.091419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:24.091419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:24.091419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:24.091419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:24.091419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:24.091419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:24.091419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:24.091419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:24.091419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:24.091419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:24.091419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:24.091419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:24.091419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:24.091419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:24.091419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:24.093768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:24.093768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:24.093768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:24.093768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:24.093768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:24.093768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:24.093768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:24.093768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:24.093768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:24.093768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:24.093768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:24.093768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:24.093768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 120/246] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:24.097750 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1ee84-8a27-420f-bb50-6e5ab4d5bc19 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:24.098716 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 127/247] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 755 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:24.112369 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ffc63-02da-4684-94df-1373a57f7f0e None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:24.160848 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:24.221486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ffc63-02da-4684-94df-1373a57f7f0e None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:24.223105 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa1ee84-8a27-420f-bb50-6e5ab4d5bc19 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:24.248946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:24.262286 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683807c9-2260-41de-961b-c94328347b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:24.263372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683807c9-2260-41de-961b-c94328347b8b None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:24.263758 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683807c9-2260-41de-961b-c94328347b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:24.264134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683807c9-2260-41de-961b-c94328347b8b None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:24.264404 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15b430-2fbc-4321-a882-77cf5ca4c4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:24.265346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15b430-2fbc-4321-a882-77cf5ca4c4a9 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:24.265758 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15b430-2fbc-4321-a882-77cf5ca4c4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:24.265866 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683807c9-2260-41de-961b-c94328347b8b None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:24.266252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15b430-2fbc-4321-a882-77cf5ca4c4a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:24.267182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15b430-2fbc-4321-a882-77cf5ca4c4a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:24.267619 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15b430-2fbc-4321-a882-77cf5ca4c4a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:24.267724 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683807c9-2260-41de-961b-c94328347b8b None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:24.269241 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15b430-2fbc-4321-a882-77cf5ca4c4a9 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:24.269813 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 128/248] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:24 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:24.276186 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683807c9-2260-41de-961b-c94328347b8b None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:24.280317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 121/249] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:24.289046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15b430-2fbc-4321-a882-77cf5ca4c4a9 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:24.398910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683807c9-2260-41de-961b-c94328347b8b None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:24.420615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15b430-2fbc-4321-a882-77cf5ca4c4a9 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:24.432795 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:24.444238 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5617d37-1a7b-4bfb-aeab-206429dfbfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:24.445014 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5617d37-1a7b-4bfb-aeab-206429dfbfc2 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:24.445362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5617d37-1a7b-4bfb-aeab-206429dfbfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:24.446135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5617d37-1a7b-4bfb-aeab-206429dfbfc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:24.447033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5617d37-1a7b-4bfb-aeab-206429dfbfc2 None admin] RBAC: Adding query filter params (role.id=784d41de5e1d415b93a1d4e0dc5c237b, scope.project.id=8855f4be124f44818348e5b04172899b, user.id=acfdecf5ccda4611b4e87060de5eb0a8) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:24.447357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5617d37-1a7b-4bfb-aeab-206429dfbfc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:24.448820 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5617d37-1a7b-4bfb-aeab-206429dfbfc2 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:24.455920 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 129/250] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:24 2022] GET /identity/v3/role_assignments?user.id=acfdecf5ccda4611b4e87060de5eb0a8&scope.project.id=8855f4be124f44818348e5b04172899b&role.id=784d41de5e1d415b93a1d4e0dc5c237b => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:24.457085 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca00583-c11f-495d-8bdd-85faeabe6ad4 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:24.457368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca00583-c11f-495d-8bdd-85faeabe6ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:24.457640 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca00583-c11f-495d-8bdd-85faeabe6ad4 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:24.463275 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 130/251] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:24.469226 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e97f61-8113-40a4-9662-e8118474fa6d None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:24.469226 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e97f61-8113-40a4-9662-e8118474fa6d None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:24.469226 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e97f61-8113-40a4-9662-e8118474fa6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:24.478418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79eb6a-465c-4389-8deb-f617d19696a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:24.479876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79eb6a-465c-4389-8deb-f617d19696a7 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:24.480157 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79eb6a-465c-4389-8deb-f617d19696a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:24.480431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79eb6a-465c-4389-8deb-f617d19696a7 None admin] PATH_INFO: `/v3/users/a5383a4dbe534b748e4240c0298eb60e` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:24.485660 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79eb6a-465c-4389-8deb-f617d19696a7 None admin] RBAC: Authorizing `identity:get_user(user_id=a5383a4dbe534b748e4240c0298eb60e)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:24.485660 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79eb6a-465c-4389-8deb-f617d19696a7 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:24.485660 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 122/252] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:24 2022] GET /identity/v3/users/a5383a4dbe534b748e4240c0298eb60e => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:24.496622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79eb6a-465c-4389-8deb-f617d19696a7 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:24.518560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-01e97f61-8113-40a4-9662-e8118474fa6d None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:24.522341 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-01e97f61-8113-40a4-9662-e8118474fa6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:24.591403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:24.608648 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 131/253] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:24.632977 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e97f61-8113-40a4-9662-e8118474fa6d None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:24.750006 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e97f61-8113-40a4-9662-e8118474fa6d None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:24.789063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc6b6f-2db9-4735-b164-18746f49a55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:24.789063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc6b6f-2db9-4735-b164-18746f49a55f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:24.789063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc6b6f-2db9-4735-b164-18746f49a55f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:24.789063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc6b6f-2db9-4735-b164-18746f49a55f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:24.801641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc6b6f-2db9-4735-b164-18746f49a55f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:24.810153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-92fc6b6f-2db9-4735-b164-18746f49a55f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:24.810153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:24.810153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:24.810153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:24.810153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:24.810153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:24.810153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:24.810153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:24.810153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:24.810153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:24.810153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:24.810153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:24.810153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:24.810153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:24.810153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:24.810153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:24.811092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:24.811092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:24.811092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:24.811092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:24.811092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:24.811092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:24.811092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:24.811092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:24.811092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:24.811092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:24.811092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:24.811092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:24.811092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:24.811092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:24.811092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:24.811092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:24.811900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:24.811900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:24.811900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:24.811900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:24.811900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:24.811900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:24.811900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:24.811900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:24.811900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:24.811900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:24.811900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:24.811900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:24.812501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 132/254] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:30:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:30:24.818130 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc6b6f-2db9-4735-b164-18746f49a55f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:24.834868 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79eb6a-465c-4389-8deb-f617d19696a7 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:24.871582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:24.893251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a4889-561a-4782-a60a-386c617ae50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:24.894166 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a4889-561a-4782-a60a-386c617ae50f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:24.894445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a4889-561a-4782-a60a-386c617ae50f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:24.894716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a4889-561a-4782-a60a-386c617ae50f None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:24.896005 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a4889-561a-4782-a60a-386c617ae50f None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:24.897601 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a4889-561a-4782-a60a-386c617ae50f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:24.904242 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 123/255] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:24 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:24.908494 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a4889-561a-4782-a60a-386c617ae50f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:24.932817 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc6b6f-2db9-4735-b164-18746f49a55f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:24.975544 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ef3c3-7182-43c7-8cca-6fccc342d9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:24.976236 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ef3c3-7182-43c7-8cca-6fccc342d9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:24.976562 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ef3c3-7182-43c7-8cca-6fccc342d9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:24.976845 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ef3c3-7182-43c7-8cca-6fccc342d9c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:24.977700 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ef3c3-7182-43c7-8cca-6fccc342d9c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:24.978012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ef3c3-7182-43c7-8cca-6fccc342d9c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:24.979811 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ef3c3-7182-43c7-8cca-6fccc342d9c6 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:24.987191 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:25.002267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 133/256] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:25.009118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ef3c3-7182-43c7-8cca-6fccc342d9c6 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:25.120377 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a4889-561a-4782-a60a-386c617ae50f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:25.184643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188edef6-85c2-40d3-a4b0-bf40489ca320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:25.185551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188edef6-85c2-40d3-a4b0-bf40489ca320 None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:25.185831 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188edef6-85c2-40d3-a4b0-bf40489ca320 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:25.186100 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188edef6-85c2-40d3-a4b0-bf40489ca320 None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b/users/a5383a4dbe534b748e4240c0298eb60e/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:25.189376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188edef6-85c2-40d3-a4b0-bf40489ca320 None admin] RBAC: Authorizing `identity:create_grant(project_id=8855f4be124f44818348e5b04172899b, user_id=a5383a4dbe534b748e4240c0298eb60e, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:25.190841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188edef6-85c2-40d3-a4b0-bf40489ca320 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:25.195315 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ef3c3-7182-43c7-8cca-6fccc342d9c6 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:25.211527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:25.223701 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 124/257] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:30:24 2022] PUT /identity/v3/projects/8855f4be124f44818348e5b04172899b/users/a5383a4dbe534b748e4240c0298eb60e/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:30:25.224981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b4031-589f-495c-8790-478c1b0129cb None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:25.225271 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b4031-589f-495c-8790-478c1b0129cb None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:25.225589 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b4031-589f-495c-8790-478c1b0129cb None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:25.232447 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 125/258] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:25.238258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee50ff-e56a-4b5e-b0e0-4d4772b80ec2 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:25.238541 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee50ff-e56a-4b5e-b0e0-4d4772b80ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:25.238817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee50ff-e56a-4b5e-b0e0-4d4772b80ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:25.277158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-baee50ff-e56a-4b5e-b0e0-4d4772b80ec2 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:25.278369 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-baee50ff-e56a-4b5e-b0e0-4d4772b80ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:25.304220 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9a91d-630a-48f1-a9d5-fa055fda03ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:25.305384 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9a91d-630a-48f1-a9d5-fa055fda03ee None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:25.305669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9a91d-630a-48f1-a9d5-fa055fda03ee None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:25.311462 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9a91d-630a-48f1-a9d5-fa055fda03ee None admin] PATH_INFO: `/v3/users/ae13961e0d9143a99555aa604e417c34` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:25.313004 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9a91d-630a-48f1-a9d5-fa055fda03ee None admin] RBAC: Authorizing `identity:get_user(user_id=ae13961e0d9143a99555aa604e417c34)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:25.314572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9a91d-630a-48f1-a9d5-fa055fda03ee None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:25.316538 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 134/259] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:25 2022] GET /identity/v3/users/ae13961e0d9143a99555aa604e417c34 => generated 283 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:25.323124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9a91d-630a-48f1-a9d5-fa055fda03ee None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:25.520565 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 126/260] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:25.556315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee50ff-e56a-4b5e-b0e0-4d4772b80ec2 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:25.627385 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:25.631371 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9a91d-630a-48f1-a9d5-fa055fda03ee None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:25.700202 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f94e04-14da-4208-afd5-cb7380e24f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:25.700870 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f94e04-14da-4208-afd5-cb7380e24f0c None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:25.701198 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f94e04-14da-4208-afd5-cb7380e24f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:25.701478 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f94e04-14da-4208-afd5-cb7380e24f0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:25.751607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee50ff-e56a-4b5e-b0e0-4d4772b80ec2 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:25.798081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:25.843748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3c6db-5715-4dad-9765-7d955256999c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:25.845295 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3c6db-5715-4dad-9765-7d955256999c None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:25.845781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3c6db-5715-4dad-9765-7d955256999c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:25.846253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3c6db-5715-4dad-9765-7d955256999c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:25.856063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f94e04-14da-4208-afd5-cb7380e24f0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:25.857731 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f94e04-14da-4208-afd5-cb7380e24f0c None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:25.875741 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3c6db-5715-4dad-9765-7d955256999c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:25.891627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-fba3c6db-5715-4dad-9765-7d955256999c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:25.891627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:25.891627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:25.891627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:25.891627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:25.891627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:25.891627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:25.891627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:25.891627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:25.891627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:25.891627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:25.891627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:25.891627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:25.891627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:25.891627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:25.891627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:25.892556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:25.892556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:25.892556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:25.892556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:25.892556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:25.892556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:25.892556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:25.892556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:25.892556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:25.892556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:25.892556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:25.892556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:25.892556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:25.892556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:25.892556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:25.892556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:25.893372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:25.893372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:25.893372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:25.893372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:25.893372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:25.893372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:25.893372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:25.893372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:25.893372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:25.893372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:25.893372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:25.893372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:25.895854 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 127/261] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:30:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:25.901731 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3c6db-5715-4dad-9765-7d955256999c None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:26.049300 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 135/262] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 727 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:26.054855 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f94e04-14da-4208-afd5-cb7380e24f0c None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:26.190596 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:26.193339 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:26.205308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3c6db-5715-4dad-9765-7d955256999c None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:26.288612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45058f-1309-4a2b-979a-ad83c3250ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:26.289616 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f94e04-14da-4208-afd5-cb7380e24f0c None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:26.290290 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45058f-1309-4a2b-979a-ad83c3250ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:26.290773 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45058f-1309-4a2b-979a-ad83c3250ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:26.291479 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45058f-1309-4a2b-979a-ad83c3250ab2 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:26.292460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45058f-1309-4a2b-979a-ad83c3250ab2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:26.292966 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45058f-1309-4a2b-979a-ad83c3250ab2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:26.302209 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45058f-1309-4a2b-979a-ad83c3250ab2 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:26.317334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 128/263] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:30:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:26.326091 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45058f-1309-4a2b-979a-ad83c3250ab2 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:26.360820 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12dd64-1938-40a5-9366-c8e4596b9a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:26.361851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12dd64-1938-40a5-9366-c8e4596b9a52 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:26.362137 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12dd64-1938-40a5-9366-c8e4596b9a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:26.362415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12dd64-1938-40a5-9366-c8e4596b9a52 None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:26.363882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12dd64-1938-40a5-9366-c8e4596b9a52 None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:26.372845 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12dd64-1938-40a5-9366-c8e4596b9a52 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:26.376197 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 136/264] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:26 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:26.391043 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12dd64-1938-40a5-9366-c8e4596b9a52 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:26.629120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12dd64-1938-40a5-9366-c8e4596b9a52 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:26.640968 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45058f-1309-4a2b-979a-ad83c3250ab2 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:26.717972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e7367-805b-4b64-8e29-7b3764b943c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:26.718659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e7367-805b-4b64-8e29-7b3764b943c9 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:26.719132 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e7367-805b-4b64-8e29-7b3764b943c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:26.719412 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e7367-805b-4b64-8e29-7b3764b943c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:26.720262 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e7367-805b-4b64-8e29-7b3764b943c9 None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=8855f4be124f44818348e5b04172899b, user.id=ae13961e0d9143a99555aa604e417c34) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:26.720571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e7367-805b-4b64-8e29-7b3764b943c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:26.722023 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e7367-805b-4b64-8e29-7b3764b943c9 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:26.737001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 137/265] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:26 2022] GET /identity/v3/role_assignments?user.id=ae13961e0d9143a99555aa604e417c34&scope.project.id=8855f4be124f44818348e5b04172899b&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 608 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:26.772722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a624f887-92fb-4def-8be1-a0df67174d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:26.773659 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624f887-92fb-4def-8be1-a0df67174d07 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:26.773946 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624f887-92fb-4def-8be1-a0df67174d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:26.774282 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a624f887-92fb-4def-8be1-a0df67174d07 None admin] PATH_INFO: `/v3/users/ae13961e0d9143a99555aa604e417c34` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:26.775542 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624f887-92fb-4def-8be1-a0df67174d07 None admin] RBAC: Authorizing `identity:get_user(user_id=ae13961e0d9143a99555aa604e417c34)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:26.777042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a624f887-92fb-4def-8be1-a0df67174d07 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:26.778718 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 129/266] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:26 2022] GET /identity/v3/users/ae13961e0d9143a99555aa604e417c34 => generated 283 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:26.784496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e7367-805b-4b64-8e29-7b3764b943c9 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:26.815412 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:26.925404 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce7c2b-5c36-440c-8282-6838764e1b9a None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:26.926334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce7c2b-5c36-440c-8282-6838764e1b9a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:26.926757 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce7c2b-5c36-440c-8282-6838764e1b9a None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:26.932697 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 130/267] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:26.938065 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0334980-0789-475d-9680-928da6778151 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:26.938654 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0334980-0789-475d-9680-928da6778151 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:26.939204 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0334980-0789-475d-9680-928da6778151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:26.972296 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-d0334980-0789-475d-9680-928da6778151 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:26.973495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d0334980-0789-475d-9680-928da6778151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:26.994448 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:27.015672 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e7367-805b-4b64-8e29-7b3764b943c9 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:27.062428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 131/268] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:27.062651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268c84f-4c0c-47b8-9d52-8754f9941870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:27.063487 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268c84f-4c0c-47b8-9d52-8754f9941870 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:27.063992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268c84f-4c0c-47b8-9d52-8754f9941870 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:27.064185 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a346da-555b-41f4-a43b-993f5323d184 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:27.064568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268c84f-4c0c-47b8-9d52-8754f9941870 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:27.064692 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a346da-555b-41f4-a43b-993f5323d184 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:27.064869 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a346da-555b-41f4-a43b-993f5323d184 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:27.066499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 132/269] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:27.072405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3be938-fc59-4f62-9299-f8e5e11b9bd4 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:27.072895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3be938-fc59-4f62-9299-f8e5e11b9bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:27.073317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3be938-fc59-4f62-9299-f8e5e11b9bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:27.098025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-cd3be938-fc59-4f62-9299-f8e5e11b9bd4 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:27.099554 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-cd3be938-fc59-4f62-9299-f8e5e11b9bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:27.161585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268c84f-4c0c-47b8-9d52-8754f9941870 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:27.164326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268c84f-4c0c-47b8-9d52-8754f9941870 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:27.182710 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:27.194056 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 133/270] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:27.196116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3be938-fc59-4f62-9299-f8e5e11b9bd4 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:27.292576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 138/271] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:27.293376 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268c84f-4c0c-47b8-9d52-8754f9941870 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:27.316163 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:27.333530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3be938-fc59-4f62-9299-f8e5e11b9bd4 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:27.374664 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d2777-fe1a-4338-b498-711f231013d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:27.375501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d2777-fe1a-4338-b498-711f231013d1 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:27.375951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d2777-fe1a-4338-b498-711f231013d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:27.376231 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d2777-fe1a-4338-b498-711f231013d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:27.386791 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d2777-fe1a-4338-b498-711f231013d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:27.393680 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268c84f-4c0c-47b8-9d52-8754f9941870 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:27.394607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-ef8d2777-fe1a-4338-b498-711f231013d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:27.394607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:27.394607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:27.394607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:27.394607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:27.394607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:27.394607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:27.394607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:27.394607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:27.394607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:27.394607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:27.394607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:27.394607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:27.394607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:27.394607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:27.394607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:27.395761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:27.395761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:27.395761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:27.395761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:27.395761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:27.395761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:27.395761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:27.395761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:27.395761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:27.395761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:27.395761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:27.395761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:27.395761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:27.395761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:27.395761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:27.395761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:27.401715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:27.401715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:27.401715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:27.401715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:27.401715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:27.401715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:27.401715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:27.401715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:27.401715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:27.401715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:27.401715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:27.401715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:27.401715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 134/272] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:27.401715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435b3a6-6ce0-45be-8003-8f2364ed8797 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:27.402423 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435b3a6-6ce0-45be-8003-8f2364ed8797 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:27.402423 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435b3a6-6ce0-45be-8003-8f2364ed8797 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:27.402423 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 135/273] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:27.402423 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435b3a6-6ce0-45be-8003-8f2364ed8797 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:27.413822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:27.441033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86972c-8293-459c-a7a8-ba82c1aee1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:27.441855 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86972c-8293-459c-a7a8-ba82c1aee1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:27.442205 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86972c-8293-459c-a7a8-ba82c1aee1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:27.442544 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86972c-8293-459c-a7a8-ba82c1aee1d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:27.443809 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a86972c-8293-459c-a7a8-ba82c1aee1d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:27.445286 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a86972c-8293-459c-a7a8-ba82c1aee1d5 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:27.451562 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 139/274] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:30:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:27.451562 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86972c-8293-459c-a7a8-ba82c1aee1d5 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:27.522571 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435b3a6-6ce0-45be-8003-8f2364ed8797 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:27.558308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1fbc3-b984-4192-8c39-b2ae375182d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:27.559205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1fbc3-b984-4192-8c39-b2ae375182d4 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:27.559311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86972c-8293-459c-a7a8-ba82c1aee1d5 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:27.559514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1fbc3-b984-4192-8c39-b2ae375182d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:27.559756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1fbc3-b984-4192-8c39-b2ae375182d4 None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:27.561045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1fbc3-b984-4192-8c39-b2ae375182d4 None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:27.562634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1fbc3-b984-4192-8c39-b2ae375182d4 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:27.564625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 136/275] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:27 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:27.565813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e82e-b633-48b5-802b-a605a8a8264d None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:27.566095 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e82e-b633-48b5-802b-a605a8a8264d None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:27.566369 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89e82e-b633-48b5-802b-a605a8a8264d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:27.584495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-be89e82e-b633-48b5-802b-a605a8a8264d None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:27.585683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-be89e82e-b633-48b5-802b-a605a8a8264d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:27.593606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:27.605892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf466f08-5ea4-403a-bf9c-468ed8d55337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:27.606568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf466f08-5ea4-403a-bf9c-468ed8d55337 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:27.606863 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf466f08-5ea4-403a-bf9c-468ed8d55337 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:27.607152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf466f08-5ea4-403a-bf9c-468ed8d55337 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:27.608325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf466f08-5ea4-403a-bf9c-468ed8d55337 None admin] RBAC: Adding query filter params (name=member) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:27.608633 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf466f08-5ea4-403a-bf9c-468ed8d55337 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:27.610167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf466f08-5ea4-403a-bf9c-468ed8d55337 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:27.618555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 140/276] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:27.623284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf466f08-5ea4-403a-bf9c-468ed8d55337 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:27.660717 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 137/277] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:27.661748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89e82e-b633-48b5-802b-a605a8a8264d None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:27.678179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:27.740027 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf466f08-5ea4-403a-bf9c-468ed8d55337 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:27.773483 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be89e82e-b633-48b5-802b-a605a8a8264d None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:27.777596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393597e-0d44-4bab-acdd-bc0e36817b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:27.778283 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393597e-0d44-4bab-acdd-bc0e36817b3a None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:27.778563 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393597e-0d44-4bab-acdd-bc0e36817b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:27.778836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393597e-0d44-4bab-acdd-bc0e36817b3a None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:27.779593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393597e-0d44-4bab-acdd-bc0e36817b3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:27.781044 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393597e-0d44-4bab-acdd-bc0e36817b3a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:27.815044 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 141/278] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:30:27 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:30:27.815044 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393597e-0d44-4bab-acdd-bc0e36817b3a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:27.824612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e33497-e8e1-4431-903e-2c1b3d604628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:27.826044 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e33497-e8e1-4431-903e-2c1b3d604628 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:27.826376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e33497-e8e1-4431-903e-2c1b3d604628 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:27.826720 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e33497-e8e1-4431-903e-2c1b3d604628 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:27.828109 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e33497-e8e1-4431-903e-2c1b3d604628 None admin] RBAC: Adding query filter params (role.id=a43ba93f358b4b1a8a445851a5a216a0, scope.project.id=8855f4be124f44818348e5b04172899b, user.id=ae13961e0d9143a99555aa604e417c34) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:27.828475 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e33497-e8e1-4431-903e-2c1b3d604628 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:27.830775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e33497-e8e1-4431-903e-2c1b3d604628 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:27.839170 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 138/279] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:27 2022] GET /identity/v3/role_assignments?user.id=ae13961e0d9143a99555aa604e417c34&scope.project.id=8855f4be124f44818348e5b04172899b&role.id=a43ba93f358b4b1a8a445851a5a216a0 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:27.844236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e33497-e8e1-4431-903e-2c1b3d604628 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:27.860768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:28.038605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:28.050116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e33497-e8e1-4431-903e-2c1b3d604628 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:28.055052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393597e-0d44-4bab-acdd-bc0e36817b3a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:28.093240 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f084c-7d4f-4cc4-b902-f6c9d516dc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:28.094419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f084c-7d4f-4cc4-b902-f6c9d516dc53 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:28.095569 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f084c-7d4f-4cc4-b902-f6c9d516dc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:28.096059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f084c-7d4f-4cc4-b902-f6c9d516dc53 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:28.112352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f084c-7d4f-4cc4-b902-f6c9d516dc53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:28.120317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622c6cc-6434-4b82-8b90-e06724a50974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:28.120469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-044f084c-7d4f-4cc4-b902-f6c9d516dc53 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:28.120469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:28.120469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:28.120469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:28.120469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:28.120469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:28.120469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:28.120469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:28.120469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:28.120469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:28.120469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:28.120469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:28.120469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:28.120469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:28.120469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:28.120469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:28.121313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:28.121313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:28.121313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:28.121313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:28.121313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:28.121313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:28.121313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:28.121313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:28.121313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:28.121313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:28.121313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:28.121313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:28.121313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:28.121313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:28.121313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:28.121313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:28.122110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:28.122110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:28.122110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:28.122110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:28.122110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:28.122110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:28.122110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:28.122110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:28.122110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:28.122110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:28.122110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:28.122110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:28.122708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622c6cc-6434-4b82-8b90-e06724a50974 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:28.122708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622c6cc-6434-4b82-8b90-e06724a50974 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:28.122708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622c6cc-6434-4b82-8b90-e06724a50974 None admin] PATH_INFO: `/v3/users/d0895d82aa5a42948f7c6ea9ede9a6d5` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:28.124704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 139/280] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:28.124957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692eca1b-a470-4725-9057-1d7b66fd192a None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:28.125256 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692eca1b-a470-4725-9057-1d7b66fd192a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:28.126332 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692eca1b-a470-4725-9057-1d7b66fd192a None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:28.127252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 140/281] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:28.132638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88252599-fac8-4663-85e6-53d59de6dec1 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:28.132638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88252599-fac8-4663-85e6-53d59de6dec1 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:28.132970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88252599-fac8-4663-85e6-53d59de6dec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:28.149895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622c6cc-6434-4b82-8b90-e06724a50974 None admin] RBAC: Authorizing `identity:get_user(user_id=d0895d82aa5a42948f7c6ea9ede9a6d5)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:28.151495 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622c6cc-6434-4b82-8b90-e06724a50974 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:28.153287 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 142/282] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:27 2022] GET /identity/v3/users/d0895d82aa5a42948f7c6ea9ede9a6d5 => generated 293 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:28.153484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-88252599-fac8-4663-85e6-53d59de6dec1 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:28.154701 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-88252599-fac8-4663-85e6-53d59de6dec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:28.154808 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622c6cc-6434-4b82-8b90-e06724a50974 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:28.230189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:28.242768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 141/283] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:28.243739 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88252599-fac8-4663-85e6-53d59de6dec1 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:28.294472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622c6cc-6434-4b82-8b90-e06724a50974 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:28.347047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147febd-b687-4af6-870d-c76e428451c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:28.347047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147febd-b687-4af6-870d-c76e428451c4 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:28.347047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147febd-b687-4af6-870d-c76e428451c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:28.347047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147febd-b687-4af6-870d-c76e428451c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:28.347444 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147febd-b687-4af6-870d-c76e428451c4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:28.351041 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147febd-b687-4af6-870d-c76e428451c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:28.351041 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147febd-b687-4af6-870d-c76e428451c4 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:28.365312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 143/284] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:28.365312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147febd-b687-4af6-870d-c76e428451c4 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:28.379041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88252599-fac8-4663-85e6-53d59de6dec1 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:28.386903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:28.435405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa270b9-fbaa-49f4-9658-033847e9b6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:28.436072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa270b9-fbaa-49f4-9658-033847e9b6fc None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:28.436352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa270b9-fbaa-49f4-9658-033847e9b6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:28.436625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa270b9-fbaa-49f4-9658-033847e9b6fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:28.437717 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa270b9-fbaa-49f4-9658-033847e9b6fc None admin] RBAC: Adding query filter params (role.id=b59c5b350c854281838a22884fd8f32f, user.id=d0895d82aa5a42948f7c6ea9ede9a6d5, scope.system=all) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:28.437828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa270b9-fbaa-49f4-9658-033847e9b6fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:28.439240 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa270b9-fbaa-49f4-9658-033847e9b6fc None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:28.445678 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 142/285] 10.222.0.44 () {56 vars in 1308 bytes} [Thu Jul 21 05:30:28 2022] GET /identity/v3/role_assignments?user.id=d0895d82aa5a42948f7c6ea9ede9a6d5&scope.system=all&role.id=b59c5b350c854281838a22884fd8f32f => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:28.446307 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa270b9-fbaa-49f4-9658-033847e9b6fc None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:28.523381 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147febd-b687-4af6-870d-c76e428451c4 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:28.568591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc094d-be91-44d7-9f6f-e23a7d553003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:28.569850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc094d-be91-44d7-9f6f-e23a7d553003 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:28.570336 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc094d-be91-44d7-9f6f-e23a7d553003 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:28.570912 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc094d-be91-44d7-9f6f-e23a7d553003 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:28.616989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc094d-be91-44d7-9f6f-e23a7d553003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:28.624466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:28.635637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-52dc094d-be91-44d7-9f6f-e23a7d553003 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:28.635637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:28.635637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:28.635637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:28.635637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:28.635637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:28.635637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:28.635637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:28.635637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:28.635637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:28.635637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:28.635637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:28.635637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:28.635637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:28.635637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:28.635637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:28.636596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:28.636596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:28.636596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:28.636596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:28.636596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:28.636596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:28.636596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:28.636596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:28.636596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:28.636596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:28.636596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:28.636596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:28.636596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:28.636596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:28.636596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:28.636596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:28.637406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:28.637406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:28.637406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:28.637406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:28.637406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:28.637406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:28.637406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:28.637406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:28.637406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:28.637406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:28.637406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:28.637406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:28.642580 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 144/286] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:28.664159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc094d-be91-44d7-9f6f-e23a7d553003 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:28.732821 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa270b9-fbaa-49f4-9658-033847e9b6fc None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:28.812956 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d38768-e76c-456d-ba3e-bdff42858072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:28.813976 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d38768-e76c-456d-ba3e-bdff42858072 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:28.814322 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d38768-e76c-456d-ba3e-bdff42858072 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:28.814669 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d38768-e76c-456d-ba3e-bdff42858072 None admin] PATH_INFO: `/v3/users/acfdecf5ccda4611b4e87060de5eb0a8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:28.816591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d38768-e76c-456d-ba3e-bdff42858072 None admin] RBAC: Authorizing `identity:get_user(user_id=acfdecf5ccda4611b4e87060de5eb0a8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:28.818150 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d38768-e76c-456d-ba3e-bdff42858072 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:28.823462 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:28.824324 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 143/287] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:28 2022] GET /identity/v3/users/acfdecf5ccda4611b4e87060de5eb0a8 => generated 297 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:28.830173 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d38768-e76c-456d-ba3e-bdff42858072 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:28.879230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc094d-be91-44d7-9f6f-e23a7d553003 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:28.959814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57daa6d6-759e-4873-b275-b2929f04efaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:28.960630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daa6d6-759e-4873-b275-b2929f04efaf None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:28.960983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daa6d6-759e-4873-b275-b2929f04efaf None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:28.961336 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daa6d6-759e-4873-b275-b2929f04efaf None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:28.962215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57daa6d6-759e-4873-b275-b2929f04efaf None admin] RBAC: Adding query filter params (name=member) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:28.962533 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57daa6d6-759e-4873-b275-b2929f04efaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:28.967045 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57daa6d6-759e-4873-b275-b2929f04efaf None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:28.982927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 145/288] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:28.993136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57daa6d6-759e-4873-b275-b2929f04efaf None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:29.038207 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d38768-e76c-456d-ba3e-bdff42858072 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:29.109662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:29.130709 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555b4dd-b791-41eb-8924-d999f22caa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:29.131957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555b4dd-b791-41eb-8924-d999f22caa89 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:29.132533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555b4dd-b791-41eb-8924-d999f22caa89 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:29.132999 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555b4dd-b791-41eb-8924-d999f22caa89 None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:29.134502 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555b4dd-b791-41eb-8924-d999f22caa89 None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:29.144778 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555b4dd-b791-41eb-8924-d999f22caa89 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:29.146877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 144/289] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:28 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:29.152865 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555b4dd-b791-41eb-8924-d999f22caa89 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:29.186987 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57daa6d6-759e-4873-b275-b2929f04efaf None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:29.251047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e7f92-932d-4c5f-9beb-6462496c4d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:29.251047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e7f92-932d-4c5f-9beb-6462496c4d2c None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:29.251047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e7f92-932d-4c5f-9beb-6462496c4d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:29.251047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e7f92-932d-4c5f-9beb-6462496c4d2c None admin] PATH_INFO: `/v3/users/a5383a4dbe534b748e4240c0298eb60e` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:29.251456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e7f92-932d-4c5f-9beb-6462496c4d2c None admin] RBAC: Authorizing `identity:get_user(user_id=a5383a4dbe534b748e4240c0298eb60e)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:29.251769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e7f92-932d-4c5f-9beb-6462496c4d2c None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:29.253746 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 146/290] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:28 2022] GET /identity/v3/users/a5383a4dbe534b748e4240c0298eb60e => generated 297 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:29.262139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e7f92-932d-4c5f-9beb-6462496c4d2c None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:29.271307 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:29.401351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555b4dd-b791-41eb-8924-d999f22caa89 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:29.440977 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e7f92-932d-4c5f-9beb-6462496c4d2c None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:29.482359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862a64d-cc85-412e-9f97-c8ad13d322f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:29.483041 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862a64d-cc85-412e-9f97-c8ad13d322f5 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:29.483319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862a64d-cc85-412e-9f97-c8ad13d322f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:29.483592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862a64d-cc85-412e-9f97-c8ad13d322f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:29.496929 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cd75f-d398-458f-8fd6-3d3426d166a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:29.497841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cd75f-d398-458f-8fd6-3d3426d166a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:29.498111 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cd75f-d398-458f-8fd6-3d3426d166a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:29.498381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cd75f-d398-458f-8fd6-3d3426d166a2 None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b/users/acfdecf5ccda4611b4e87060de5eb0a8/roles/784d41de5e1d415b93a1d4e0dc5c237b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:29.503032 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2cd75f-d398-458f-8fd6-3d3426d166a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8855f4be124f44818348e5b04172899b, user_id=acfdecf5ccda4611b4e87060de5eb0a8, role_id=784d41de5e1d415b93a1d4e0dc5c237b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:29.503032 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2cd75f-d398-458f-8fd6-3d3426d166a2 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:29.535947 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 145/291] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:30:29 2022] PUT /identity/v3/projects/8855f4be124f44818348e5b04172899b/users/acfdecf5ccda4611b4e87060de5eb0a8/roles/784d41de5e1d415b93a1d4e0dc5c237b => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:30:29.582784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862a64d-cc85-412e-9f97-c8ad13d322f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:29.584364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862a64d-cc85-412e-9f97-c8ad13d322f5 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:29.588075 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:29.699422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 147/292] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:29.702185 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cd75f-d398-458f-8fd6-3d3426d166a2 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:29.825302 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:29.888447 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cd75f-d398-458f-8fd6-3d3426d166a2 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:29.930661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60189e64-4406-4069-b436-cbbc38d17b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:29.931542 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60189e64-4406-4069-b436-cbbc38d17b0b None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:29.931824 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60189e64-4406-4069-b436-cbbc38d17b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:29.935051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60189e64-4406-4069-b436-cbbc38d17b0b None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:29.935051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60189e64-4406-4069-b436-cbbc38d17b0b None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:29.935051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60189e64-4406-4069-b436-cbbc38d17b0b None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:29.937130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 146/293] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:29 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:29.945356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862a64d-cc85-412e-9f97-c8ad13d322f5 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:30.154805 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862a64d-cc85-412e-9f97-c8ad13d322f5 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:30.166650 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:30.252471 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0dc6a8-13e5-4f53-b845-ba7dec1d0274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:30.253079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dc6a8-13e5-4f53-b845-ba7dec1d0274 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:30.253363 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dc6a8-13e5-4f53-b845-ba7dec1d0274 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:30.253635 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0dc6a8-13e5-4f53-b845-ba7dec1d0274 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:30.254479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0dc6a8-13e5-4f53-b845-ba7dec1d0274 None admin] RBAC: Adding query filter params (role.id=b59c5b350c854281838a22884fd8f32f, scope.project.id=8855f4be124f44818348e5b04172899b, user.id=a5383a4dbe534b748e4240c0298eb60e) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:30.254790 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0dc6a8-13e5-4f53-b845-ba7dec1d0274 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:30.256305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0dc6a8-13e5-4f53-b845-ba7dec1d0274 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:30.273457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 148/294] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:29 2022] GET /identity/v3/role_assignments?user.id=a5383a4dbe534b748e4240c0298eb60e&scope.project.id=8855f4be124f44818348e5b04172899b&role.id=b59c5b350c854281838a22884fd8f32f => generated 608 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:30.570201 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe44dd-5492-4c35-9343-071bdaf97041 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:30.570694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe44dd-5492-4c35-9343-071bdaf97041 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:30.570694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe44dd-5492-4c35-9343-071bdaf97041 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:30.572171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 147/295] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:30.580747 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fbd03-04af-4c07-88d0-dca20b26f4e0 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:30.581236 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fbd03-04af-4c07-88d0-dca20b26f4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:30.581334 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fbd03-04af-4c07-88d0-dca20b26f4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:30.601837 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-5e4fbd03-04af-4c07-88d0-dca20b26f4e0 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:30.603463 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5e4fbd03-04af-4c07-88d0-dca20b26f4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:30.683253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 149/296] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:30.702294 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe44dd-5492-4c35-9343-071bdaf97041 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:30.821540 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe44dd-5492-4c35-9343-071bdaf97041 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:30.824727 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47530e0-34b5-4ed2-a307-33edac2a8bbc None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:30.825414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47530e0-34b5-4ed2-a307-33edac2a8bbc None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:30.825414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47530e0-34b5-4ed2-a307-33edac2a8bbc None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:30.834548 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 150/297] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:30 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:30:30.840657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672cb5c-8d5c-44be-af8b-124ef4db288e None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:30.841191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672cb5c-8d5c-44be-af8b-124ef4db288e None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:30.841659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672cb5c-8d5c-44be-af8b-124ef4db288e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:30.866822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-0672cb5c-8d5c-44be-af8b-124ef4db288e None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:30.868125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0672cb5c-8d5c-44be-af8b-124ef4db288e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:30.873720 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:30.892265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2230f8-6e80-44ea-b8d9-47c305bfbe88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:30.892265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2230f8-6e80-44ea-b8d9-47c305bfbe88 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:30.892265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2230f8-6e80-44ea-b8d9-47c305bfbe88 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:30.892265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2230f8-6e80-44ea-b8d9-47c305bfbe88 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:30.909238 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2230f8-6e80-44ea-b8d9-47c305bfbe88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:30.919348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-1e2230f8-6e80-44ea-b8d9-47c305bfbe88 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:30.919348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:30.919348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:30.919348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:30.919348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:30.919348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:30.919348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:30.919348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:30.919348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:30.919348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:30.919348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:30.919348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:30.919348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:30.919348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:30.919348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:30.919348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:30.920301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:30.920301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:30.920301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:30.920301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:30.920301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:30.920301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:30.920301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:30.920301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:30.920301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:30.920301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:30.920301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:30.920301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:30.920301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:30.920301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:30.920301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:30.920301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:30.921083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:30.921083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:30.921083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:30.921083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:30.921083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:30.921083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:30.921083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:30.921083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:30.921083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:30.921083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:30.921083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:30.921083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:30.923294 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 148/298] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:30.933322 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2230f8-6e80-44ea-b8d9-47c305bfbe88 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:30.973589 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 151/299] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:31.006916 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672cb5c-8d5c-44be-af8b-124ef4db288e None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:31.046455 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:31.055151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2230f8-6e80-44ea-b8d9-47c305bfbe88 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:31.104011 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936db28d-a1f6-4b98-bb9d-c160667f02d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:31.104933 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936db28d-a1f6-4b98-bb9d-c160667f02d7 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:31.105218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936db28d-a1f6-4b98-bb9d-c160667f02d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:31.105492 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936db28d-a1f6-4b98-bb9d-c160667f02d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:31.106245 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936db28d-a1f6-4b98-bb9d-c160667f02d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:31.106557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936db28d-a1f6-4b98-bb9d-c160667f02d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:31.108123 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936db28d-a1f6-4b98-bb9d-c160667f02d7 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:31.122265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 149/300] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:31.122265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1c2cd-5838-4be8-af8a-4c472fc25c07 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:31.122265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1c2cd-5838-4be8-af8a-4c472fc25c07 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:31.122265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1c2cd-5838-4be8-af8a-4c472fc25c07 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:31.122265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 150/301] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:31.123232 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1c2cd-5838-4be8-af8a-4c472fc25c07 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:31.136435 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672cb5c-8d5c-44be-af8b-124ef4db288e None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:31.177921 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8706e90-2560-4b15-ae18-077a8cb7ebee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:31.180486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:31.180602 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8706e90-2560-4b15-ae18-077a8cb7ebee None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:31.180602 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8706e90-2560-4b15-ae18-077a8cb7ebee None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:31.180602 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8706e90-2560-4b15-ae18-077a8cb7ebee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:31.183496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8706e90-2560-4b15-ae18-077a8cb7ebee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:31.184328 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8706e90-2560-4b15-ae18-077a8cb7ebee None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:31.186163 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 152/302] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:30:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:31.192697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f186ffe-b855-4b69-8726-ba446913ead7 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:31.192990 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f186ffe-b855-4b69-8726-ba446913ead7 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:31.193273 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f186ffe-b855-4b69-8726-ba446913ead7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:31.210261 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-7f186ffe-b855-4b69-8726-ba446913ead7 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:31.211499 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7f186ffe-b855-4b69-8726-ba446913ead7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:31.248961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1c2cd-5838-4be8-af8a-4c472fc25c07 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:31.289375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c1a03-27cc-4e70-be83-0878577541cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:31.290399 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c1a03-27cc-4e70-be83-0878577541cd None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:31.290710 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c1a03-27cc-4e70-be83-0878577541cd None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:31.290995 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c1a03-27cc-4e70-be83-0878577541cd None admin] PATH_INFO: `/v3/users/c255372ff9064fd2a138a51226393614` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:31.296740 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 153/303] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:31.297413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f186ffe-b855-4b69-8726-ba446913ead7 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:31.306670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c1a03-27cc-4e70-be83-0878577541cd None admin] RBAC: Authorizing `identity:get_user(user_id=c255372ff9064fd2a138a51226393614)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:31.308259 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c1a03-27cc-4e70-be83-0878577541cd None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:31.310137 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 151/304] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:31 2022] GET /identity/v3/users/c255372ff9064fd2a138a51226393614 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:31.316514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c1a03-27cc-4e70-be83-0878577541cd None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:31.335166 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:31.425892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f186ffe-b855-4b69-8726-ba446913ead7 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:31.440132 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c1a03-27cc-4e70-be83-0878577541cd None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:31.444495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:31.470536 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af3d3e-a517-4eb0-affa-7115e33be839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:31.471420 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af3d3e-a517-4eb0-affa-7115e33be839 None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:31.471779 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af3d3e-a517-4eb0-affa-7115e33be839 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:31.472199 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af3d3e-a517-4eb0-affa-7115e33be839 None admin] PATH_INFO: `/v3/groups` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:31.473471 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af3d3e-a517-4eb0-affa-7115e33be839 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:31.475299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af3d3e-a517-4eb0-affa-7115e33be839 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:31.482859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d875a7-ddcf-45ff-83b5-1fd1b4c551f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:31.483722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d875a7-ddcf-45ff-83b5-1fd1b4c551f3 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:31.484211 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d875a7-ddcf-45ff-83b5-1fd1b4c551f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:31.484614 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d875a7-ddcf-45ff-83b5-1fd1b4c551f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:31.485516 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d875a7-ddcf-45ff-83b5-1fd1b4c551f3 None admin] RBAC: Adding query filter params (role.id=784d41de5e1d415b93a1d4e0dc5c237b, user.id=c255372ff9064fd2a138a51226393614, scope.system=all) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:31.485895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d875a7-ddcf-45ff-83b5-1fd1b4c551f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:31.489385 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d875a7-ddcf-45ff-83b5-1fd1b4c551f3 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:31.503043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 152/305] 10.222.0.44 () {56 vars in 1308 bytes} [Thu Jul 21 05:30:31 2022] GET /identity/v3/role_assignments?user.id=c255372ff9064fd2a138a51226393614&scope.system=all&role.id=784d41de5e1d415b93a1d4e0dc5c237b => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:31.503043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d875a7-ddcf-45ff-83b5-1fd1b4c551f3 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:31.512007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 154/306] 10.222.0.44 () {60 vars in 1134 bytes} [Thu Jul 21 05:30:31 2022] POST /identity/v3/groups => generated 229 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:30:31.637926 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d875a7-ddcf-45ff-83b5-1fd1b4c551f3 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:31.695127 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80cc58c-e8bf-4807-937a-f3c52618d15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:31.695127 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80cc58c-e8bf-4807-937a-f3c52618d15f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:31.695127 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80cc58c-e8bf-4807-937a-f3c52618d15f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:31.695127 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80cc58c-e8bf-4807-937a-f3c52618d15f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:31.708023 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80cc58c-e8bf-4807-937a-f3c52618d15f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:31.717632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-e80cc58c-e8bf-4807-937a-f3c52618d15f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:31.717632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:31.717632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:31.717632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:31.717632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:31.717632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:31.717632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:31.717632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:31.717632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:31.717632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:31.717632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:31.717632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:31.717632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:31.717632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:31.717632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:31.717632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:31.718628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:31.718628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:31.718628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:31.718628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:31.718628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:31.718628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:31.718628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:31.718628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:31.718628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:31.718628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:31.718628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:31.718628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:31.718628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:31.718628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:31.718628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:31.718628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:31.719469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:31.719469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:31.719469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:31.719469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:31.719469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:31.719469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:31.719469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:31.719469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:31.719469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:31.719469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:31.719469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:31.719469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:31.719469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 153/307] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:31.726346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af3d3e-a517-4eb0-affa-7115e33be839 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:31.843368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:31.859030 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af3d3e-a517-4eb0-affa-7115e33be839 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:31.895687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b13aa5-f87b-43ea-b2b3-b2f20029f7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:31.896394 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b13aa5-f87b-43ea-b2b3-b2f20029f7f2 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:31.896684 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b13aa5-f87b-43ea-b2b3-b2f20029f7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:31.896967 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b13aa5-f87b-43ea-b2b3-b2f20029f7f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:31.897729 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b13aa5-f87b-43ea-b2b3-b2f20029f7f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:31.898040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b13aa5-f87b-43ea-b2b3-b2f20029f7f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:31.899542 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b13aa5-f87b-43ea-b2b3-b2f20029f7f2 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:31.913324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 155/308] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:31.914866 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80cc58c-e8bf-4807-937a-f3c52618d15f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:31.972145 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:32.063437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80cc58c-e8bf-4807-937a-f3c52618d15f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:32.082788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c42de-221f-4372-b875-95e40e95125b None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:32.083161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c42de-221f-4372-b875-95e40e95125b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:32.083378 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c42de-221f-4372-b875-95e40e95125b None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:32.085119 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 156/309] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:32.100619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d66631-2217-44aa-8272-abcb07567d35 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:32.100914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d66631-2217-44aa-8272-abcb07567d35 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:32.101185 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d66631-2217-44aa-8272-abcb07567d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:32.110224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c752a-37e7-49fc-8b13-522489a3e2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:32.111482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c752a-37e7-49fc-8b13-522489a3e2dc None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:32.111968 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c752a-37e7-49fc-8b13-522489a3e2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:32.112440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c752a-37e7-49fc-8b13-522489a3e2dc None admin] PATH_INFO: `/v3/users/d0895d82aa5a42948f7c6ea9ede9a6d5` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:32.113884 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c752a-37e7-49fc-8b13-522489a3e2dc None admin] RBAC: Authorizing `identity:get_user(user_id=d0895d82aa5a42948f7c6ea9ede9a6d5)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:32.115883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c752a-37e7-49fc-8b13-522489a3e2dc None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:32.118393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 154/310] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:31 2022] GET /identity/v3/users/d0895d82aa5a42948f7c6ea9ede9a6d5 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:32.124239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c752a-37e7-49fc-8b13-522489a3e2dc None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:32.135733 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-22d66631-2217-44aa-8272-abcb07567d35 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:32.137251 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-22d66631-2217-44aa-8272-abcb07567d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:32.242528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 157/311] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:32.271722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d66631-2217-44aa-8272-abcb07567d35 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:32.303397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:32.311298 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c752a-37e7-49fc-8b13-522489a3e2dc None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:32.371832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8fce3f-6468-4f04-9784-2115db8025b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:32.372709 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fce3f-6468-4f04-9784-2115db8025b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:32.372985 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fce3f-6468-4f04-9784-2115db8025b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:32.373254 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8fce3f-6468-4f04-9784-2115db8025b0 None admin] PATH_INFO: `/v3/system/users/d0895d82aa5a42948f7c6ea9ede9a6d5/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:32.376842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8fce3f-6468-4f04-9784-2115db8025b0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d0895d82aa5a42948f7c6ea9ede9a6d5, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:32.378236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8fce3f-6468-4f04-9784-2115db8025b0 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:32.401245 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 155/312] 10.222.0.44 () {58 vars in 1334 bytes} [Thu Jul 21 05:30:32 2022] PUT /identity/v3/system/users/d0895d82aa5a42948f7c6ea9ede9a6d5/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:30:32.420521 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:32.449633 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d66631-2217-44aa-8272-abcb07567d35 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:32.489462 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e90bc-5051-4aae-bfee-255cfdde48ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:32.490728 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e90bc-5051-4aae-bfee-255cfdde48ca None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:32.491283 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e90bc-5051-4aae-bfee-255cfdde48ca None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:32.491772 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e90bc-5051-4aae-bfee-255cfdde48ca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:32.504617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e90bc-5051-4aae-bfee-255cfdde48ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:32.513144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-ab3e90bc-5051-4aae-bfee-255cfdde48ca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:32.513144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:32.513144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:32.513144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:32.513144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:32.513144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:32.513144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:32.513144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:32.513144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:32.513144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:32.513144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:32.513144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:32.513144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:32.513144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:32.513144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:32.513144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:32.514116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:32.514116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:32.514116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:32.514116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:32.514116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:32.514116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:32.514116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:32.514116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:32.514116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:32.514116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:32.514116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:32.514116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:32.514116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:32.514116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:32.514116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:32.514116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:32.514903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:32.514903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:32.514903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:32.514903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:32.514903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:32.514903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:32.514903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:32.514903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:32.514903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:32.514903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:32.514903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:32.514903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:32.517846 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 158/313] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:30:32.526047 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8fce3f-6468-4f04-9784-2115db8025b0 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:32.696844 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8fce3f-6468-4f04-9784-2115db8025b0 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:32.734890 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b57ed4-a4c8-49d6-a684-64016b4d7f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:32.736025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b57ed4-a4c8-49d6-a684-64016b4d7f88 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:32.736510 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b57ed4-a4c8-49d6-a684-64016b4d7f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:32.737000 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b57ed4-a4c8-49d6-a684-64016b4d7f88 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:32.738040 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b57ed4-a4c8-49d6-a684-64016b4d7f88 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:32.738564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b57ed4-a4c8-49d6-a684-64016b4d7f88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:32.740416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b57ed4-a4c8-49d6-a684-64016b4d7f88 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:32.752061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 156/314] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:32.757559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e90bc-5051-4aae-bfee-255cfdde48ca None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:32.866101 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e90bc-5051-4aae-bfee-255cfdde48ca None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:32.901869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c2135-a97c-4dfa-a8e0-cff025c1c316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:32.903251 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c2135-a97c-4dfa-a8e0-cff025c1c316 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:32.903823 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c2135-a97c-4dfa-a8e0-cff025c1c316 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:32.904294 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c2135-a97c-4dfa-a8e0-cff025c1c316 None admin] PATH_INFO: `/v3/users/acfdecf5ccda4611b4e87060de5eb0a8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:32.906014 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c2135-a97c-4dfa-a8e0-cff025c1c316 None admin] RBAC: Authorizing `identity:get_user(user_id=acfdecf5ccda4611b4e87060de5eb0a8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:32.907787 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c2135-a97c-4dfa-a8e0-cff025c1c316 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:32.909832 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 159/315] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:32 2022] GET /identity/v3/users/acfdecf5ccda4611b4e87060de5eb0a8 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:32.919060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b57ed4-a4c8-49d6-a684-64016b4d7f88 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:32.960531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:33.056640 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b57ed4-a4c8-49d6-a684-64016b4d7f88 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:33.094491 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d123e95-9be3-406b-ae0b-7149272a0de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:33.095594 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d123e95-9be3-406b-ae0b-7149272a0de7 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:33.096076 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d123e95-9be3-406b-ae0b-7149272a0de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:33.096555 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d123e95-9be3-406b-ae0b-7149272a0de7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:33.171239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d123e95-9be3-406b-ae0b-7149272a0de7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:33.173091 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d123e95-9be3-406b-ae0b-7149272a0de7 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:33.201237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:33.344640 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 157/316] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:33.352358 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c2135-a97c-4dfa-a8e0-cff025c1c316 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:33.366109 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:33.502239 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c2135-a97c-4dfa-a8e0-cff025c1c316 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:33.538825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd4919-79e1-446c-9479-91183ae36e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:33.539905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd4919-79e1-446c-9479-91183ae36e1b None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:33.540111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd4919-79e1-446c-9479-91183ae36e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:33.540396 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd4919-79e1-446c-9479-91183ae36e1b None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:33.541596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd4919-79e1-446c-9479-91183ae36e1b None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:33.543100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd4919-79e1-446c-9479-91183ae36e1b None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:33.545129 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 160/317] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:33 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:33.551780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d123e95-9be3-406b-ae0b-7149272a0de7 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:33.688733 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d123e95-9be3-406b-ae0b-7149272a0de7 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:33.715473 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:33.728058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a1b12-eded-4766-aa40-4327753c25ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:33.731072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a1b12-eded-4766-aa40-4327753c25ed None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:33.731072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a1b12-eded-4766-aa40-4327753c25ed None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:33.731072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a1b12-eded-4766-aa40-4327753c25ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:33.731072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a1b12-eded-4766-aa40-4327753c25ed None admin] RBAC: Adding query filter params (role.id=784d41de5e1d415b93a1d4e0dc5c237b, scope.project.id=8855f4be124f44818348e5b04172899b, user.id=acfdecf5ccda4611b4e87060de5eb0a8) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:33.731072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a1b12-eded-4766-aa40-4327753c25ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:33.732164 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a1b12-eded-4766-aa40-4327753c25ed None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:33.738710 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 158/318] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:30:33 2022] GET /identity/v3/role_assignments?user.id=acfdecf5ccda4611b4e87060de5eb0a8&scope.project.id=8855f4be124f44818348e5b04172899b&role.id=784d41de5e1d415b93a1d4e0dc5c237b => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:33.767971 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd4224-65b9-4014-a1fd-5de2dfa0e6f2 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:33.768945 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd4224-65b9-4014-a1fd-5de2dfa0e6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:33.769423 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd4224-65b9-4014-a1fd-5de2dfa0e6f2 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:33.771255 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 161/319] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:33.777713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13ba2c-42d9-4123-9104-e034e8379e7e None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:33.778312 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13ba2c-42d9-4123-9104-e034e8379e7e None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:33.778787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13ba2c-42d9-4123-9104-e034e8379e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:33.800733 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-bb13ba2c-42d9-4123-9104-e034e8379e7e None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:33.802335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-bb13ba2c-42d9-4123-9104-e034e8379e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:33.895927 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 159/320] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:33.922038 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd4224-65b9-4014-a1fd-5de2dfa0e6f2 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:34.000398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e55437-5597-4bdc-b176-5f3b3c2c901c None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:34.000767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e55437-5597-4bdc-b176-5f3b3c2c901c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:34.001529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e55437-5597-4bdc-b176-5f3b3c2c901c None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:34.002784 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 160/321] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:34.008937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70107912-17c8-49ec-95f4-87cb6d7aec8b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:34.009251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70107912-17c8-49ec-95f4-87cb6d7aec8b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:34.009540 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70107912-17c8-49ec-95f4-87cb6d7aec8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:34.028043 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:34.042607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-70107912-17c8-49ec-95f4-87cb6d7aec8b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:34.044341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-70107912-17c8-49ec-95f4-87cb6d7aec8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:34.064700 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd4224-65b9-4014-a1fd-5de2dfa0e6f2 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:34.104289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bedee4-4a33-4ceb-a592-05f209a39f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:34.105201 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bedee4-4a33-4ceb-a592-05f209a39f0d None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:34.105483 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bedee4-4a33-4ceb-a592-05f209a39f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:34.105867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bedee4-4a33-4ceb-a592-05f209a39f0d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:34.119438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bedee4-4a33-4ceb-a592-05f209a39f0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:34.127765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-03bedee4-4a33-4ceb-a592-05f209a39f0d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:34.127765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:34.127765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:34.127765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:34.127765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:34.127765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:34.127765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:34.127765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:34.127765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:34.127765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:34.127765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:34.127765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:34.127765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:34.127765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:34.127765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:34.127765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:34.128694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:34.128694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:34.128694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:34.128694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:34.128694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:34.128694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:34.128694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:34.128694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:34.128694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:34.128694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:34.128694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:34.128694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:34.128694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:34.128694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:34.128694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:34.128694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:34.134936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:34.134936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:34.134936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:34.134936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:34.134936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:34.134936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:34.134936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:34.134936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:34.134936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:34.134936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:34.134936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:34.134936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:34.134936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 162/322] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:34.136217 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bedee4-4a33-4ceb-a592-05f209a39f0d None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:34.143326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 161/323] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:34.160526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70107912-17c8-49ec-95f4-87cb6d7aec8b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:34.227703 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:34.276386 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bedee4-4a33-4ceb-a592-05f209a39f0d None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:34.297067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70107912-17c8-49ec-95f4-87cb6d7aec8b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:34.313920 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bf5eb-a4d7-437b-828f-510391b0dd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:34.314613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bf5eb-a4d7-437b-828f-510391b0dd9b None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:34.314894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bf5eb-a4d7-437b-828f-510391b0dd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:34.315206 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bf5eb-a4d7-437b-828f-510391b0dd9b None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:34.315964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bf5eb-a4d7-437b-828f-510391b0dd9b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:34.316264 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bf5eb-a4d7-437b-828f-510391b0dd9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:34.317661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3bf5eb-a4d7-437b-828f-510391b0dd9b None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:34.319293 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:34.331992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 163/324] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:34.337541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bf5eb-a4d7-437b-828f-510391b0dd9b None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:34.348595 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e0888-2b4d-4026-b260-921e2851b77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:34.349942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e0888-2b4d-4026-b260-921e2851b77b None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:34.350236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e0888-2b4d-4026-b260-921e2851b77b None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:34.350516 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e0888-2b4d-4026-b260-921e2851b77b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:34.351785 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e0888-2b4d-4026-b260-921e2851b77b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:34.353992 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e0888-2b4d-4026-b260-921e2851b77b None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:34.356541 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 162/325] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:30:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:34.362085 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e0888-2b4d-4026-b260-921e2851b77b None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:34.462413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3bf5eb-a4d7-437b-828f-510391b0dd9b None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:34.497900 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e0888-2b4d-4026-b260-921e2851b77b None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:34.513431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816a582-0153-418d-ab39-75a8716e68ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:34.514340 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816a582-0153-418d-ab39-75a8716e68ca None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:34.514631 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816a582-0153-418d-ab39-75a8716e68ca None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:34.514912 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816a582-0153-418d-ab39-75a8716e68ca None admin] PATH_INFO: `/v3/users/c255372ff9064fd2a138a51226393614` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:34.516328 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816a582-0153-418d-ab39-75a8716e68ca None admin] RBAC: Authorizing `identity:get_user(user_id=c255372ff9064fd2a138a51226393614)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:34.517810 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816a582-0153-418d-ab39-75a8716e68ca None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:34.517920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:34.523055 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 164/326] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:34 2022] GET /identity/v3/users/c255372ff9064fd2a138a51226393614 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:34.525759 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816a582-0153-418d-ab39-75a8716e68ca None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:34.550957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640953a6-8c67-404a-bd20-c125ec0a4a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:34.551885 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640953a6-8c67-404a-bd20-c125ec0a4a05 None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:34.552233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640953a6-8c67-404a-bd20-c125ec0a4a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:34.552579 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640953a6-8c67-404a-bd20-c125ec0a4a05 None admin] PATH_INFO: `/v3/groups` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:34.553346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640953a6-8c67-404a-bd20-c125ec0a4a05 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:34.554858 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640953a6-8c67-404a-bd20-c125ec0a4a05 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:34.575363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 163/327] 10.222.0.44 () {60 vars in 1134 bytes} [Thu Jul 21 05:30:34 2022] POST /identity/v3/groups => generated 226 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:30:34.602542 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:34.663921 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816a582-0153-418d-ab39-75a8716e68ca None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:34.671829 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d83f2-b5cf-45c5-8b9c-62c5d1927204 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:34.672173 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d83f2-b5cf-45c5-8b9c-62c5d1927204 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:34.672396 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d83f2-b5cf-45c5-8b9c-62c5d1927204 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:34.674037 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 164/328] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:30:34.679740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023ba73-7806-4b70-9104-7855097fadd9 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:34.680101 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023ba73-7806-4b70-9104-7855097fadd9 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:34.680416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023ba73-7806-4b70-9104-7855097fadd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:34.705209 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-f023ba73-7806-4b70-9104-7855097fadd9 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:34.705970 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01de25-a1d4-4987-abae-fc2c716c669c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:34.706419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f023ba73-7806-4b70-9104-7855097fadd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:34.707420 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01de25-a1d4-4987-abae-fc2c716c669c None admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:34.708014 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01de25-a1d4-4987-abae-fc2c716c669c None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:34.708491 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01de25-a1d4-4987-abae-fc2c716c669c None admin] PATH_INFO: `/v3/system/users/c255372ff9064fd2a138a51226393614/roles/784d41de5e1d415b93a1d4e0dc5c237b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:34.710451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01de25-a1d4-4987-abae-fc2c716c669c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c255372ff9064fd2a138a51226393614, role_id=784d41de5e1d415b93a1d4e0dc5c237b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:34.713032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01de25-a1d4-4987-abae-fc2c716c669c None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:34.726600 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 165/329] 10.222.0.44 () {58 vars in 1334 bytes} [Thu Jul 21 05:30:34 2022] PUT /identity/v3/system/users/c255372ff9064fd2a138a51226393614/roles/784d41de5e1d415b93a1d4e0dc5c237b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:30:34.817525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 165/330] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:34.841400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01de25-a1d4-4987-abae-fc2c716c669c None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:34.983373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01de25-a1d4-4987-abae-fc2c716c669c None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:35.013004 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:35.022866 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b075ab-1af6-439d-ab0f-1262450872b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:35.023651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b075ab-1af6-439d-ab0f-1262450872b6 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:35.023874 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b075ab-1af6-439d-ab0f-1262450872b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:35.024116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b075ab-1af6-439d-ab0f-1262450872b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:35.036223 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b075ab-1af6-439d-ab0f-1262450872b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:35.050719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-84b075ab-1af6-439d-ab0f-1262450872b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:35.050719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:35.050719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:35.050719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:35.050719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:35.050719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:35.050719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:35.050719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:35.050719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:35.050719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:35.050719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:35.050719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:35.050719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:35.050719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:35.050719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:35.050719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:35.051786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:35.051786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:35.051786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:35.051786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:35.051786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:35.051786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:35.051786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:35.051786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:35.051786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:35.051786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:35.051786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:35.051786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:35.051786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:35.051786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:35.051786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:35.051786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:35.052591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:35.052591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:35.052591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:35.052591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:35.052591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:35.052591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:35.052591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:35.052591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:35.052591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:35.052591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:35.052591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:35.052591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:35.052591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 166/331] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:35.059121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023ba73-7806-4b70-9104-7855097fadd9 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:35.185515 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023ba73-7806-4b70-9104-7855097fadd9 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:35.189615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:35.230084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08501c4-e928-4d23-a173-c10a79e21e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:35.230796 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08501c4-e928-4d23-a173-c10a79e21e60 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:35.231119 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08501c4-e928-4d23-a173-c10a79e21e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:35.231404 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08501c4-e928-4d23-a173-c10a79e21e60 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:35.232179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08501c4-e928-4d23-a173-c10a79e21e60 None admin] RBAC: Adding query filter params (name=member) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:35.232489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08501c4-e928-4d23-a173-c10a79e21e60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:35.234117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08501c4-e928-4d23-a173-c10a79e21e60 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:35.249861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 166/332] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:35.262657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b075ab-1af6-439d-ab0f-1262450872b6 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:35.389989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b075ab-1af6-439d-ab0f-1262450872b6 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:35.435829 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d519265-28dc-42ee-8177-162f7784b0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:35.439035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d519265-28dc-42ee-8177-162f7784b0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:35.439035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d519265-28dc-42ee-8177-162f7784b0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:35.439035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d519265-28dc-42ee-8177-162f7784b0f9 None admin] PATH_INFO: `/v3/users/d0895d82aa5a42948f7c6ea9ede9a6d5` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:35.439035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d519265-28dc-42ee-8177-162f7784b0f9 None admin] RBAC: Authorizing `identity:get_user(user_id=d0895d82aa5a42948f7c6ea9ede9a6d5)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:35.441783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d519265-28dc-42ee-8177-162f7784b0f9 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:35.441783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 167/333] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:35 2022] GET /identity/v3/users/d0895d82aa5a42948f7c6ea9ede9a6d5 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:35.448564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08501c4-e928-4d23-a173-c10a79e21e60 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:35.567434 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08501c4-e928-4d23-a173-c10a79e21e60 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:35.607359 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e8db9-c2db-496b-ab8e-085af661644a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:35.608038 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e8db9-c2db-496b-ab8e-085af661644a None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:35.608328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e8db9-c2db-496b-ab8e-085af661644a None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:35.608599 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e8db9-c2db-496b-ab8e-085af661644a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:35.609454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e8db9-c2db-496b-ab8e-085af661644a None admin] RBAC: Adding query filter params (role.id=b59c5b350c854281838a22884fd8f32f, user.id=d0895d82aa5a42948f7c6ea9ede9a6d5, scope.system=all) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:35.609773 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e8db9-c2db-496b-ab8e-085af661644a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:35.611228 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e8db9-c2db-496b-ab8e-085af661644a None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:35.624675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 167/334] 10.222.0.44 () {56 vars in 1308 bytes} [Thu Jul 21 05:30:35 2022] GET /identity/v3/role_assignments?user.id=d0895d82aa5a42948f7c6ea9ede9a6d5&scope.system=all&role.id=b59c5b350c854281838a22884fd8f32f => generated 510 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:37.555382 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4e371-712d-4694-ad13-eb3fff5cbb1d None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:37.556001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4e371-712d-4694-ad13-eb3fff5cbb1d None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:37.556001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4e371-712d-4694-ad13-eb3fff5cbb1d None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:37.557451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 168/335] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:37.565281 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759af443-812b-4887-8d47-e2c586e969a1 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:37.565281 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759af443-812b-4887-8d47-e2c586e969a1 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:37.565281 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759af443-812b-4887-8d47-e2c586e969a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:37.584026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-759af443-812b-4887-8d47-e2c586e969a1 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:37.585220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-759af443-812b-4887-8d47-e2c586e969a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:37.592979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb387bc-2f86-457f-a776-d3e81435e200 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:37.593261 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb387bc-2f86-457f-a776-d3e81435e200 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:37.593535 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb387bc-2f86-457f-a776-d3e81435e200 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:37.595197 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 169/336] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:37.606037 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:37.609843 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f077f-bce1-40da-8c3d-dd62bf81018b None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:37.610128 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f077f-bce1-40da-8c3d-dd62bf81018b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:37.610408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f077f-bce1-40da-8c3d-dd62bf81018b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:37.651661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-824f077f-bce1-40da-8c3d-dd62bf81018b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:37.652876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-824f077f-bce1-40da-8c3d-dd62bf81018b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:37.692607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 168/337] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:37.733890 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759af443-812b-4887-8d47-e2c586e969a1 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:37.792341 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:37.836398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 170/338] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:37.837738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bcb9a-250a-482c-98be-0d8719c4021f None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:37.838023 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bcb9a-250a-482c-98be-0d8719c4021f None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:37.838294 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bcb9a-250a-482c-98be-0d8719c4021f None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:37.840816 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 171/339] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:37.858764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8be9e-0933-4d09-bdf4-c2ed3dbf5962 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:37.859337 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8be9e-0933-4d09-bdf4-c2ed3dbf5962 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:37.860332 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8be9e-0933-4d09-bdf4-c2ed3dbf5962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:37.898400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-72f8be9e-0933-4d09-bdf4-c2ed3dbf5962 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:37.900867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-72f8be9e-0933-4d09-bdf4-c2ed3dbf5962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:38.006126 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 172/340] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:38.008479 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759af443-812b-4887-8d47-e2c586e969a1 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:38.010010 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8be9e-0933-4d09-bdf4-c2ed3dbf5962 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:38.095840 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b6399-30b4-4b90-bf2c-b29c6dc63ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:38.096926 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b6399-30b4-4b90-bf2c-b29c6dc63ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:38.097343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b6399-30b4-4b90-bf2c-b29c6dc63ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:38.097766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b6399-30b4-4b90-bf2c-b29c6dc63ef3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:38.137700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b6399-30b4-4b90-bf2c-b29c6dc63ef3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:38.145896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-8f7b6399-30b4-4b90-bf2c-b29c6dc63ef3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:38.145896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:38.145896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:38.145896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:38.145896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:38.145896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:38.145896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:38.145896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:38.145896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:38.145896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:38.145896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:38.145896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:38.145896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:38.145896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:38.145896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:38.145896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:38.146832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:38.146832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:38.146832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:38.146832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:38.146832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:38.146832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:38.146832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:38.146832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:38.146832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:38.146832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:38.146832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:38.146832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:38.146832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:38.146832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:38.146832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:38.146832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:38.147684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:38.147684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:38.147684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:38.147684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:38.147684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:38.147684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:38.147684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:38.147684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:38.147684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:38.147684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:38.147684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:38.147684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:38.150359 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 169/341] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:38.151518 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b6399-30b4-4b90-bf2c-b29c6dc63ef3 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:38.190437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8be9e-0933-4d09-bdf4-c2ed3dbf5962 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:38.216906 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:38.223400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:38.253278 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd21d8-ac9b-4e49-a351-cbcef252b5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:38.254110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd21d8-ac9b-4e49-a351-cbcef252b5cb None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:38.254627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd21d8-ac9b-4e49-a351-cbcef252b5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:38.254905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd21d8-ac9b-4e49-a351-cbcef252b5cb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:38.276235 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd21d8-ac9b-4e49-a351-cbcef252b5cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:38.295189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-fbfd21d8-ac9b-4e49-a351-cbcef252b5cb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:38.295189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:38.295189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:38.295189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:38.295189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:38.295189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:38.295189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:38.295189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:38.295189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:38.295189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:38.295189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:38.295189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:38.295189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:38.295189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:38.295189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:38.295189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:38.296270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:38.296270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:38.296270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:38.296270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:38.296270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:38.296270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:38.296270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:38.296270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:38.296270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:38.296270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:38.296270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:38.296270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:38.296270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:38.296270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:38.296270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:38.296270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:38.297086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:38.297086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:38.297086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:38.297086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:38.297086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:38.297086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:38.297086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:38.297086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:38.297086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:38.297086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:38.297086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 21 05:30:38.297086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:38.301864 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 173/342] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:38.302859 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd21d8-ac9b-4e49-a351-cbcef252b5cb None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:38.418281 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b6399-30b4-4b90-bf2c-b29c6dc63ef3 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:38.507944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af0668-814a-445f-ba1f-abad615afa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:38.509153 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af0668-814a-445f-ba1f-abad615afa46 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:38.509599 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af0668-814a-445f-ba1f-abad615afa46 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:38.510008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af0668-814a-445f-ba1f-abad615afa46 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:38.522568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd21d8-ac9b-4e49-a351-cbcef252b5cb None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:38.527954 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af0668-814a-445f-ba1f-abad615afa46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:38.546244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-64af0668-814a-445f-ba1f-abad615afa46 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:38.546244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:38.546244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:38.546244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:38.546244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:38.546244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:38.546244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:38.546244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:38.546244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:38.546244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:38.546244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:38.546244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:38.546244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:38.546244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:38.546244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:38.546244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:38.547371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:38.547371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:38.547371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:38.547371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:38.547371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:38.547371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:38.547371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:38.547371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:38.547371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:38.547371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:38.547371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:38.547371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:38.547371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:38.547371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:38.547371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:38.547371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:38.548180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:38.548180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:38.548180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:38.548180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:38.548180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:38.548180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:38.548180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:38.548180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:38.548180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:38.548180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:38.548180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:38.548180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:38.553740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 170/343] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:30:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:38.555004 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af0668-814a-445f-ba1f-abad615afa46 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:38.612688 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c6002-ca16-4545-93c8-8913da122e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:38.614048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c6002-ca16-4545-93c8-8913da122e01 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:38.616697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c6002-ca16-4545-93c8-8913da122e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:38.617149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c6002-ca16-4545-93c8-8913da122e01 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:38.618272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c6002-ca16-4545-93c8-8913da122e01 None admin] RBAC: Adding query filter params (name=member) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:38.620413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c6002-ca16-4545-93c8-8913da122e01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:38.622257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c6002-ca16-4545-93c8-8913da122e01 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:38.637718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 174/344] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:38.639463 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c6002-ca16-4545-93c8-8913da122e01 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:38.686846 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:38.706155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:38.736297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af0668-814a-445f-ba1f-abad615afa46 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:38.754577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c6002-ca16-4545-93c8-8913da122e01 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:38.783750 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef6bdf-4fa3-494d-845d-ee2c7050f9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:38.784720 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef6bdf-4fa3-494d-845d-ee2c7050f9fa None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:38.785150 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef6bdf-4fa3-494d-845d-ee2c7050f9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:38.785564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef6bdf-4fa3-494d-845d-ee2c7050f9fa None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:38.786515 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef6bdf-4fa3-494d-845d-ee2c7050f9fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:38.787007 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef6bdf-4fa3-494d-845d-ee2c7050f9fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:38.789028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef6bdf-4fa3-494d-845d-ee2c7050f9fa None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:38.803874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 171/345] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:38.805942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1a31f-afbb-4c9e-8992-d7272a8f21da None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:38.806440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1a31f-afbb-4c9e-8992-d7272a8f21da None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:38.807216 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1a31f-afbb-4c9e-8992-d7272a8f21da None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:38.808973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 172/346] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:38.809917 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1a31f-afbb-4c9e-8992-d7272a8f21da None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:38.815079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debc265-f121-492a-9eb4-ee3e963fd0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:38.815079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debc265-f121-492a-9eb4-ee3e963fd0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:38.815079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debc265-f121-492a-9eb4-ee3e963fd0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:38.815079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debc265-f121-492a-9eb4-ee3e963fd0f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:38.815443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debc265-f121-492a-9eb4-ee3e963fd0f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:38.815443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debc265-f121-492a-9eb4-ee3e963fd0f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:38.816202 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1debc265-f121-492a-9eb4-ee3e963fd0f1 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:38.829891 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 175/347] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:30:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:38.830868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debc265-f121-492a-9eb4-ee3e963fd0f1 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:38.946793 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1a31f-afbb-4c9e-8992-d7272a8f21da None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:38.947725 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:38.951814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1debc265-f121-492a-9eb4-ee3e963fd0f1 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:38.986061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da7b5d-a9f0-49b8-a10f-4dfd81e438b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:38.987075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da7b5d-a9f0-49b8-a10f-4dfd81e438b4 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:38.987609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da7b5d-a9f0-49b8-a10f-4dfd81e438b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:38.988075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da7b5d-a9f0-49b8-a10f-4dfd81e438b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:38.992621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:38.992985 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1f34a-d521-48cb-8b6a-a1761db095fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:38.993953 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1f34a-d521-48cb-8b6a-a1761db095fa None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:38.994825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1f34a-d521-48cb-8b6a-a1761db095fa None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:38.994825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1f34a-d521-48cb-8b6a-a1761db095fa None admin] PATH_INFO: `/v3/users/c255372ff9064fd2a138a51226393614` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:38.996021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1f34a-d521-48cb-8b6a-a1761db095fa None admin] RBAC: Authorizing `identity:get_user(user_id=c255372ff9064fd2a138a51226393614)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:38.999350 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1f34a-d521-48cb-8b6a-a1761db095fa None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:39.000872 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 176/348] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:30:38 2022] GET /identity/v3/users/c255372ff9064fd2a138a51226393614 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:39.002094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5080661-19dc-4e25-9e53-144e5932ebd3 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:39.002381 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5080661-19dc-4e25-9e53-144e5932ebd3 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:39.002751 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5080661-19dc-4e25-9e53-144e5932ebd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:39.020594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-c5080661-19dc-4e25-9e53-144e5932ebd3 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:39.021795 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-c5080661-19dc-4e25-9e53-144e5932ebd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:39.081207 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da7b5d-a9f0-49b8-a10f-4dfd81e438b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:39.082456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da7b5d-a9f0-49b8-a10f-4dfd81e438b4 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:39.125487 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 177/349] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:39.126684 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5080661-19dc-4e25-9e53-144e5932ebd3 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:39.238761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 173/350] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:39.240019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da7b5d-a9f0-49b8-a10f-4dfd81e438b4 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:39.269939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:39.273513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5080661-19dc-4e25-9e53-144e5932ebd3 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:39.296645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:39.328060 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10aca7c-4004-4913-ae7c-8b8b35a35c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:39.329045 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10aca7c-4004-4913-ae7c-8b8b35a35c24 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:39.329503 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10aca7c-4004-4913-ae7c-8b8b35a35c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:39.329945 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10aca7c-4004-4913-ae7c-8b8b35a35c24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:39.404895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da7b5d-a9f0-49b8-a10f-4dfd81e438b4 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:39.419421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10aca7c-4004-4913-ae7c-8b8b35a35c24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:39.421310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10aca7c-4004-4913-ae7c-8b8b35a35c24 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:39.457230 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53baacc-b826-4ff2-b534-cbb2c761d20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:39.457902 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53baacc-b826-4ff2-b534-cbb2c761d20b None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:39.458185 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53baacc-b826-4ff2-b534-cbb2c761d20b None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:39.458460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53baacc-b826-4ff2-b534-cbb2c761d20b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:39.460136 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53baacc-b826-4ff2-b534-cbb2c761d20b None admin] RBAC: Adding query filter params (role.id=784d41de5e1d415b93a1d4e0dc5c237b, user.id=c255372ff9064fd2a138a51226393614, scope.system=all) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:39.460136 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53baacc-b826-4ff2-b534-cbb2c761d20b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:39.464941 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53baacc-b826-4ff2-b534-cbb2c761d20b None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:39.479977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 174/351] 10.222.0.44 () {56 vars in 1308 bytes} [Thu Jul 21 05:30:39 2022] GET /identity/v3/role_assignments?user.id=c255372ff9064fd2a138a51226393614&scope.system=all&role.id=784d41de5e1d415b93a1d4e0dc5c237b => generated 510 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:39.482292 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01035c0-fa89-4be6-9d57-78ef96f6c7de None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:39.482583 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01035c0-fa89-4be6-9d57-78ef96f6c7de None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:39.482864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01035c0-fa89-4be6-9d57-78ef96f6c7de None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:39.488690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 175/352] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:39.489826 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01035c0-fa89-4be6-9d57-78ef96f6c7de None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:39.561586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:39.573097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 178/353] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:39.573864 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10aca7c-4004-4913-ae7c-8b8b35a35c24 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:39.644743 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:39.659478 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01035c0-fa89-4be6-9d57-78ef96f6c7de None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:39.711725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b518cf-67d9-4962-9cad-8b29ed7f110b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:39.712556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b518cf-67d9-4962-9cad-8b29ed7f110b None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:39.712904 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b518cf-67d9-4962-9cad-8b29ed7f110b None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:39.713116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b518cf-67d9-4962-9cad-8b29ed7f110b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:39.732716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b518cf-67d9-4962-9cad-8b29ed7f110b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:39.742670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-66b518cf-67d9-4962-9cad-8b29ed7f110b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:39.742670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:39.742670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:39.742670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:39.742670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:39.742670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:39.742670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:39.742670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:39.742670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:39.742670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:39.742670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:39.742670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:39.742670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:39.742670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:39.742670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:39.742670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:39.743872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:39.743872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:39.743872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:39.743872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:39.743872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:39.743872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:39.743872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:39.743872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:39.743872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:39.743872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:39.743872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:39.743872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:39.743872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:39.743872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:39.743872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:39.743872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:39.744754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:39.744754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:39.744754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:39.744754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:39.744754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:39.744754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:39.744754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:39.744754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:39.744754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:39.744754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:39.744754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:39.744754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:39.746441 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 176/354] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:39.748668 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10aca7c-4004-4913-ae7c-8b8b35a35c24 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:39.749120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba63cd-e305-428d-861e-633b823f7dfd None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:39.749401 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba63cd-e305-428d-861e-633b823f7dfd None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:39.749694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba63cd-e305-428d-861e-633b823f7dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:39.773608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-00ba63cd-e305-428d-861e-633b823f7dfd None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:39.774896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-00ba63cd-e305-428d-861e-633b823f7dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:39.821437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514a0fa-c151-4969-9463-e041f594e6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:39.822334 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514a0fa-c151-4969-9463-e041f594e6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:39.822615 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514a0fa-c151-4969-9463-e041f594e6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:39.822942 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514a0fa-c151-4969-9463-e041f594e6a0 None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:39.827144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514a0fa-c151-4969-9463-e041f594e6a0 None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:39.829951 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9514a0fa-c151-4969-9463-e041f594e6a0 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:39.830611 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 179/355] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:39 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:39.831366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514a0fa-c151-4969-9463-e041f594e6a0 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:39.893192 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 177/356] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:39.893880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba63cd-e305-428d-861e-633b823f7dfd None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:39.930522 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:39.988794 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9514a0fa-c151-4969-9463-e041f594e6a0 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:40.017858 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:40.039779 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fc9e0-ab88-4ab9-aa32-d6f5de571774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:40.040662 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fc9e0-ab88-4ab9-aa32-d6f5de571774 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:40.040946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fc9e0-ab88-4ab9-aa32-d6f5de571774 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:40.041213 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925fc9e0-ab88-4ab9-aa32-d6f5de571774 None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:40.043051 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925fc9e0-ab88-4ab9-aa32-d6f5de571774 None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:40.043690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ba63cd-e305-428d-861e-633b823f7dfd None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:40.045150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925fc9e0-ab88-4ab9-aa32-d6f5de571774 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:40.047872 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 180/357] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:39 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:40.048685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fc9e0-ab88-4ab9-aa32-d6f5de571774 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:40.096177 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64843618-713c-4cb8-9469-85731e9bbf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:40.096879 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64843618-713c-4cb8-9469-85731e9bbf13 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:40.097164 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64843618-713c-4cb8-9469-85731e9bbf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:40.097437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64843618-713c-4cb8-9469-85731e9bbf13 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:40.098218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64843618-713c-4cb8-9469-85731e9bbf13 None admin] RBAC: Adding query filter params (name=member) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:40.098520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64843618-713c-4cb8-9469-85731e9bbf13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:40.100864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64843618-713c-4cb8-9469-85731e9bbf13 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:40.110877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 178/358] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:40.112926 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbeb8a1-47de-4b7e-a253-f12b34b659c5 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:40.113219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbeb8a1-47de-4b7e-a253-f12b34b659c5 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:40.113501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbeb8a1-47de-4b7e-a253-f12b34b659c5 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:40.114803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 179/359] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:40.115696 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbeb8a1-47de-4b7e-a253-f12b34b659c5 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:40.214898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925fc9e0-ab88-4ab9-aa32-d6f5de571774 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:40.265658 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17644ba-c55b-416c-8d07-85b05ae139e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:40.266540 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17644ba-c55b-416c-8d07-85b05ae139e6 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:40.266825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17644ba-c55b-416c-8d07-85b05ae139e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:40.267206 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17644ba-c55b-416c-8d07-85b05ae139e6 None admin] PATH_INFO: `/v3/groups/082eef03ba5f4329ab39e13dc4e287e2` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:40.269902 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:40.278194 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbeb8a1-47de-4b7e-a253-f12b34b659c5 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:40.279791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17644ba-c55b-416c-8d07-85b05ae139e6 None admin] RBAC: Authorizing `identity:get_group(group_id=082eef03ba5f4329ab39e13dc4e287e2)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:40.281321 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17644ba-c55b-416c-8d07-85b05ae139e6 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:40.291151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 181/360] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jul 21 05:30:40 2022] GET /identity/v3/groups/082eef03ba5f4329ab39e13dc4e287e2 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:40.291151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17644ba-c55b-416c-8d07-85b05ae139e6 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:40.331077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90ef81-43ad-4ab9-9b69-74d9bb564923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:40.333353 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90ef81-43ad-4ab9-9b69-74d9bb564923 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:40.333854 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90ef81-43ad-4ab9-9b69-74d9bb564923 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:40.334328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90ef81-43ad-4ab9-9b69-74d9bb564923 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:40.352506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90ef81-43ad-4ab9-9b69-74d9bb564923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:40.361564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-ee90ef81-43ad-4ab9-9b69-74d9bb564923 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:40.361564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:40.361564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:40.361564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:40.361564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:40.361564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:40.361564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:40.361564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:40.361564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:40.361564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:40.361564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:40.361564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:40.361564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:40.361564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:40.361564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:40.361564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:40.362523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:40.362523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:40.362523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:40.362523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:40.362523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:40.362523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:40.362523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:40.362523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:40.362523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:40.362523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:40.362523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:40.362523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:40.362523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:40.362523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:40.362523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:40.362523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:40.363436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:40.363436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:40.363436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:40.363436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:40.363436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:40.363436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:40.363436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:40.363436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:40.363436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:40.363436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:40.363436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:40.363436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:40.363436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 180/361] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:30:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:40.364662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90ef81-43ad-4ab9-9b69-74d9bb564923 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:40.378762 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:40.449040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17644ba-c55b-416c-8d07-85b05ae139e6 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:40.501606 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a7a3c-2d8a-45a3-8df1-f01a0194d734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:40.502334 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a7a3c-2d8a-45a3-8df1-f01a0194d734 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:40.502617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a7a3c-2d8a-45a3-8df1-f01a0194d734 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:40.502887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a7a3c-2d8a-45a3-8df1-f01a0194d734 None admin] PATH_INFO: `/v3/groups/082eef03ba5f4329ab39e13dc4e287e2` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:40.504213 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a7a3c-2d8a-45a3-8df1-f01a0194d734 None admin] RBAC: Authorizing `identity:get_group(group_id=082eef03ba5f4329ab39e13dc4e287e2)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:40.505754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a7a3c-2d8a-45a3-8df1-f01a0194d734 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:40.507718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 182/362] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jul 21 05:30:40 2022] GET /identity/v3/groups/082eef03ba5f4329ab39e13dc4e287e2 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:40.509861 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9f1e8-9cfc-4b02-990f-33f1a12b2aff None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:40.509861 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9f1e8-9cfc-4b02-990f-33f1a12b2aff None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:40.509861 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9f1e8-9cfc-4b02-990f-33f1a12b2aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:40.532917 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-5eb9f1e8-9cfc-4b02-990f-33f1a12b2aff None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:40.534599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5eb9f1e8-9cfc-4b02-990f-33f1a12b2aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:40.539993 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90ef81-43ad-4ab9-9b69-74d9bb564923 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:40.596495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da84c14-7495-40a8-b46a-39d64b8a4d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:40.597182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da84c14-7495-40a8-b46a-39d64b8a4d9c None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:40.597465 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da84c14-7495-40a8-b46a-39d64b8a4d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:40.597754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da84c14-7495-40a8-b46a-39d64b8a4d9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:40.639868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:40.656141 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 183/363] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:40.657099 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9f1e8-9cfc-4b02-990f-33f1a12b2aff None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:40.692603 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da84c14-7495-40a8-b46a-39d64b8a4d9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:40.694324 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da84c14-7495-40a8-b46a-39d64b8a4d9c None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:40.704948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:40.814735 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9f1e8-9cfc-4b02-990f-33f1a12b2aff None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:40.859984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 181/364] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:40.860501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da84c14-7495-40a8-b46a-39d64b8a4d9c None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:40.874026 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5a89e-a546-4976-bec2-49c663e7d4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:40.874026 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5a89e-a546-4976-bec2-49c663e7d4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:40.875856 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5a89e-a546-4976-bec2-49c663e7d4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:40.875856 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5a89e-a546-4976-bec2-49c663e7d4b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:40.876361 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5a89e-a546-4976-bec2-49c663e7d4b0 None admin] RBAC: Adding query filter params (group.id=082eef03ba5f4329ab39e13dc4e287e2, role.id=b59c5b350c854281838a22884fd8f32f, scope.project.id=db74f3f0803a45359025fb277815314b) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:40.876677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5a89e-a546-4976-bec2-49c663e7d4b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:40.878500 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea5a89e-a546-4976-bec2-49c663e7d4b0 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:40.887766 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 184/365] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jul 21 05:30:40 2022] GET /identity/v3/role_assignments?group.id=082eef03ba5f4329ab39e13dc4e287e2&scope.project.id=db74f3f0803a45359025fb277815314b&role.id=b59c5b350c854281838a22884fd8f32f => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:40.888620 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5a89e-a546-4976-bec2-49c663e7d4b0 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:40.981589 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:41.032671 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da84c14-7495-40a8-b46a-39d64b8a4d9c None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:41.034845 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5a89e-a546-4976-bec2-49c663e7d4b0 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:41.043886 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:41.080434 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34135891-e2f1-4e9d-9589-de4b6f14a8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:41.081136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34135891-e2f1-4e9d-9589-de4b6f14a8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:41.081423 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34135891-e2f1-4e9d-9589-de4b6f14a8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:41.081700 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34135891-e2f1-4e9d-9589-de4b6f14a8a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:41.082589 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34135891-e2f1-4e9d-9589-de4b6f14a8a0 None admin] RBAC: Adding query filter params (group.id=082eef03ba5f4329ab39e13dc4e287e2, role.id=a43ba93f358b4b1a8a445851a5a216a0, scope.project.id=db74f3f0803a45359025fb277815314b) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:41.082920 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34135891-e2f1-4e9d-9589-de4b6f14a8a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:41.084087 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6a333-1e24-4ae0-9bf5-66617a1c6817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:41.084772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6a333-1e24-4ae0-9bf5-66617a1c6817 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:41.085083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6a333-1e24-4ae0-9bf5-66617a1c6817 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:41.085182 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34135891-e2f1-4e9d-9589-de4b6f14a8a0 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:41.085347 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6a333-1e24-4ae0-9bf5-66617a1c6817 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:41.086088 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6a333-1e24-4ae0-9bf5-66617a1c6817 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:41.086464 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6a333-1e24-4ae0-9bf5-66617a1c6817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:41.088061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d6a333-1e24-4ae0-9bf5-66617a1c6817 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:41.092382 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 185/366] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jul 21 05:30:40 2022] GET /identity/v3/role_assignments?group.id=082eef03ba5f4329ab39e13dc4e287e2&scope.project.id=db74f3f0803a45359025fb277815314b&role.id=a43ba93f358b4b1a8a445851a5a216a0 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:41.093211 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34135891-e2f1-4e9d-9589-de4b6f14a8a0 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:41.098366 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 182/367] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:30:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:41.099685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6a333-1e24-4ae0-9bf5-66617a1c6817 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:41.255052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34135891-e2f1-4e9d-9589-de4b6f14a8a0 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:41.289523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6a333-1e24-4ae0-9bf5-66617a1c6817 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:41.292965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:41.327858 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58266b-5f05-41b6-ae96-c5adace8a690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:41.328839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58266b-5f05-41b6-ae96-c5adace8a690 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:41.329195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58266b-5f05-41b6-ae96-c5adace8a690 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:41.329543 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58266b-5f05-41b6-ae96-c5adace8a690 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:41.347110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58266b-5f05-41b6-ae96-c5adace8a690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:41.351890 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c75fe6-ce1c-4205-9a2f-b06500f9e1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:41.352335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-4e58266b-5f05-41b6-ae96-c5adace8a690 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:41.352335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:41.352335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:41.352335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:41.352335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:41.352335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:41.352335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:41.352335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:41.352335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:41.352335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:41.352335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:41.352335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:41.352335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:41.352335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:41.352335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:41.352335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:41.353193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:41.353193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:41.353193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:41.353193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:41.353193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:41.353193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:41.353193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:41.353193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:41.353193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:41.353193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:41.353193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:41.353193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:41.353193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:41.353193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:41.353193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:41.353193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:41.353979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:41.353979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:41.353979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:41.353979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:41.353979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:41.353979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:41.353979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:41.353979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:41.353979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:41.353979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:41.353979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:41.353979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:41.354588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c75fe6-ce1c-4205-9a2f-b06500f9e1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:41.354588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c75fe6-ce1c-4205-9a2f-b06500f9e1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:41.354588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c75fe6-ce1c-4205-9a2f-b06500f9e1e4 None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:41.354766 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 186/368] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:30:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:41.354766 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58266b-5f05-41b6-ae96-c5adace8a690 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:41.357458 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c75fe6-ce1c-4205-9a2f-b06500f9e1e4 None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:41.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c75fe6-ce1c-4205-9a2f-b06500f9e1e4 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:41.361101 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 183/369] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:41 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:41.361859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c75fe6-ce1c-4205-9a2f-b06500f9e1e4 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:41.432643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:41.499048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58266b-5f05-41b6-ae96-c5adace8a690 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:41.554964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c75fe6-ce1c-4205-9a2f-b06500f9e1e4 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:41.559035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d26ed-6c77-4fa3-832f-b1f4a889fe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:41.559035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d26ed-6c77-4fa3-832f-b1f4a889fe71 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:41.559035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d26ed-6c77-4fa3-832f-b1f4a889fe71 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:41.559035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d26ed-6c77-4fa3-832f-b1f4a889fe71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:41.647116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae3050-bffa-4457-8579-8248e271169d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:41.647116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae3050-bffa-4457-8579-8248e271169d None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:41.647116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae3050-bffa-4457-8579-8248e271169d None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:41.647116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae3050-bffa-4457-8579-8248e271169d None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:41.647507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae3050-bffa-4457-8579-8248e271169d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:41.647507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae3050-bffa-4457-8579-8248e271169d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:41.647507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae3050-bffa-4457-8579-8248e271169d None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:41.668813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 184/370] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:41.668813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae3050-bffa-4457-8579-8248e271169d None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:41.673995 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:41.703047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d26ed-6c77-4fa3-832f-b1f4a889fe71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:41.703255 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d26ed-6c77-4fa3-832f-b1f4a889fe71 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:41.871046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 187/371] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:41.872911 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d26ed-6c77-4fa3-832f-b1f4a889fe71 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:41.874696 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:41.891048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae3050-bffa-4457-8579-8248e271169d None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:41.973684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de7aa5-13d5-4793-ba99-8ce226d85c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:41.975952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de7aa5-13d5-4793-ba99-8ce226d85c74 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:41.976843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de7aa5-13d5-4793-ba99-8ce226d85c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:41.977128 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de7aa5-13d5-4793-ba99-8ce226d85c74 None admin] PATH_INFO: `/v3/groups/082eef03ba5f4329ab39e13dc4e287e2` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:41.981614 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de7aa5-13d5-4793-ba99-8ce226d85c74 None admin] RBAC: Authorizing `identity:get_group(group_id=082eef03ba5f4329ab39e13dc4e287e2)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:41.985731 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de7aa5-13d5-4793-ba99-8ce226d85c74 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:41.989852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 185/372] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jul 21 05:30:41 2022] GET /identity/v3/groups/082eef03ba5f4329ab39e13dc4e287e2 => generated 226 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:41.990868 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de7aa5-13d5-4793-ba99-8ce226d85c74 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:42.067039 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:42.107045 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d26ed-6c77-4fa3-832f-b1f4a889fe71 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:42.180385 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6bfc5-4e19-4ec3-b357-fb534a247a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:42.187035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6bfc5-4e19-4ec3-b357-fb534a247a8e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:42.187035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6bfc5-4e19-4ec3-b357-fb534a247a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:42.187035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6bfc5-4e19-4ec3-b357-fb534a247a8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:42.207045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de7aa5-13d5-4793-ba99-8ce226d85c74 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:42.280568 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8babc-26f1-4d53-b224-7239084ab494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:42.287042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8babc-26f1-4d53-b224-7239084ab494 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:42.287042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8babc-26f1-4d53-b224-7239084ab494 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:42.287042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8babc-26f1-4d53-b224-7239084ab494 None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:42.291033 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd8babc-26f1-4d53-b224-7239084ab494 None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:42.291033 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd8babc-26f1-4d53-b224-7239084ab494 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:42.299038 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 186/373] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:41 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:42.299038 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8babc-26f1-4d53-b224-7239084ab494 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:42.323037 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6bfc5-4e19-4ec3-b357-fb534a247a8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:42.327037 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6bfc5-4e19-4ec3-b357-fb534a247a8e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:42.371036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:42.483035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:42.493093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8babc-26f1-4d53-b224-7239084ab494 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:42.515034 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 188/374] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:42.515034 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6bfc5-4e19-4ec3-b357-fb534a247a8e None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:42.559131 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17b388-6a62-495f-884d-5660bd5d2fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:42.559131 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17b388-6a62-495f-884d-5660bd5d2fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:42.559131 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17b388-6a62-495f-884d-5660bd5d2fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:42.559131 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17b388-6a62-495f-884d-5660bd5d2fa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:42.559509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17b388-6a62-495f-884d-5660bd5d2fa8 None admin] RBAC: Adding query filter params (group.id=082eef03ba5f4329ab39e13dc4e287e2, role.id=b59c5b350c854281838a22884fd8f32f, scope.project.id=8855f4be124f44818348e5b04172899b) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:42.559666 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17b388-6a62-495f-884d-5660bd5d2fa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:42.563033 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17b388-6a62-495f-884d-5660bd5d2fa8 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:42.575034 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 187/375] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jul 21 05:30:42 2022] GET /identity/v3/role_assignments?group.id=082eef03ba5f4329ab39e13dc4e287e2&scope.project.id=8855f4be124f44818348e5b04172899b&role.id=b59c5b350c854281838a22884fd8f32f => generated 260 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:42.575034 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17b388-6a62-495f-884d-5660bd5d2fa8 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:42.800649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6bfc5-4e19-4ec3-b357-fb534a247a8e None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:42.867115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79d928-267b-498a-ab55-770a3e447183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:42.867115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79d928-267b-498a-ab55-770a3e447183 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:42.867115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79d928-267b-498a-ab55-770a3e447183 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:42.867115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79d928-267b-498a-ab55-770a3e447183 None admin] PATH_INFO: `/v3/groups/082eef03ba5f4329ab39e13dc4e287e2` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:42.871568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79d928-267b-498a-ab55-770a3e447183 None admin] RBAC: Authorizing `identity:get_group(group_id=082eef03ba5f4329ab39e13dc4e287e2)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:42.871568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79d928-267b-498a-ab55-770a3e447183 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:42.883033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 189/376] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jul 21 05:30:42 2022] GET /identity/v3/groups/082eef03ba5f4329ab39e13dc4e287e2 => generated 226 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:42.883033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79d928-267b-498a-ab55-770a3e447183 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:42.907044 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:42.912050 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17b388-6a62-495f-884d-5660bd5d2fa8 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:43.007051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498227a-014a-4d1c-913e-f4b5cbc90399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:43.007051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498227a-014a-4d1c-913e-f4b5cbc90399 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:43.007051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498227a-014a-4d1c-913e-f4b5cbc90399 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:43.007051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498227a-014a-4d1c-913e-f4b5cbc90399 None admin] PATH_INFO: `/v3/projects/f7ec186b03924ec6a68eab5b0ea7d82d` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:43.011052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498227a-014a-4d1c-913e-f4b5cbc90399 None admin] RBAC: Authorizing `identity:get_project(project_id=f7ec186b03924ec6a68eab5b0ea7d82d)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:43.012609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498227a-014a-4d1c-913e-f4b5cbc90399 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:43.018787 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:43.018931 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 188/377] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:42 2022] GET /identity/v3/projects/f7ec186b03924ec6a68eab5b0ea7d82d => generated 344 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:43.023034 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498227a-014a-4d1c-913e-f4b5cbc90399 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:43.131161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79d928-267b-498a-ab55-770a3e447183 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:43.225258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de880c-2104-4859-a13d-79cb4193b22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:43.227852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de880c-2104-4859-a13d-79cb4193b22e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:43.228526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de880c-2104-4859-a13d-79cb4193b22e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:43.231046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de880c-2104-4859-a13d-79cb4193b22e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:43.231046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de880c-2104-4859-a13d-79cb4193b22e None admin] RBAC: Adding query filter params (group.id=082eef03ba5f4329ab39e13dc4e287e2, role.id=a43ba93f358b4b1a8a445851a5a216a0, scope.project.id=8855f4be124f44818348e5b04172899b) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:43.231046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de880c-2104-4859-a13d-79cb4193b22e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:43.233576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de880c-2104-4859-a13d-79cb4193b22e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:43.242465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 190/378] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jul 21 05:30:42 2022] GET /identity/v3/role_assignments?group.id=082eef03ba5f4329ab39e13dc4e287e2&scope.project.id=8855f4be124f44818348e5b04172899b&role.id=a43ba93f358b4b1a8a445851a5a216a0 => generated 260 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:43.248586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a8ec3-691a-41b1-b9c5-73a718c81223 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:43.249717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a8ec3-691a-41b1-b9c5-73a718c81223 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:43.250723 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a8ec3-691a-41b1-b9c5-73a718c81223 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:43.255334 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 191/379] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:43 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:43.268952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9829128-9173-491f-bf26-405d723ef4e0 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:43.269342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9829128-9173-491f-bf26-405d723ef4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:43.269728 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9829128-9173-491f-bf26-405d723ef4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:43.312520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498227a-014a-4d1c-913e-f4b5cbc90399 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:43.327081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-b9829128-9173-491f-bf26-405d723ef4e0 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:43.327081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b9829128-9173-491f-bf26-405d723ef4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:43.446848 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12108808-9b24-479f-8436-4968ae57be15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:43.449948 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 192/380] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:43.450088 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12108808-9b24-479f-8436-4968ae57be15 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:43.455068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12108808-9b24-479f-8436-4968ae57be15 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:43.455068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12108808-9b24-479f-8436-4968ae57be15 None admin] PATH_INFO: `/v3/groups/7527d489bc5f4d629f998c4b9f2d8d4b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:43.460589 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47511c0d-6897-462b-9adf-004995b9bc70 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:43.461181 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47511c0d-6897-462b-9adf-004995b9bc70 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:43.461680 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47511c0d-6897-462b-9adf-004995b9bc70 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:43.467809 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 193/381] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:43.479050 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf977d76-3678-4247-aab3-1365791a2441 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:43.479050 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf977d76-3678-4247-aab3-1365791a2441 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:43.479050 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf977d76-3678-4247-aab3-1365791a2441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:43.487755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12108808-9b24-479f-8436-4968ae57be15 None admin] RBAC: Authorizing `identity:get_group(group_id=7527d489bc5f4d629f998c4b9f2d8d4b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:43.489530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12108808-9b24-479f-8436-4968ae57be15 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:43.496113 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 189/382] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jul 21 05:30:43 2022] GET /identity/v3/groups/7527d489bc5f4d629f998c4b9f2d8d4b => generated 229 bytes in 472 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:43.501611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12108808-9b24-479f-8436-4968ae57be15 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:43.516714 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-cf977d76-3678-4247-aab3-1365791a2441 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:43.519257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-cf977d76-3678-4247-aab3-1365791a2441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:43.569961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:43.579398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:43.596334 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 194/383] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:43.597197 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf977d76-3678-4247-aab3-1365791a2441 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:43.627037 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12108808-9b24-479f-8436-4968ae57be15 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:43.665580 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffdeb69-b6c4-4a4e-9c93-ca894a5ac169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:43.666572 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdeb69-b6c4-4a4e-9c93-ca894a5ac169 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:43.667004 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdeb69-b6c4-4a4e-9c93-ca894a5ac169 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:43.667451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdeb69-b6c4-4a4e-9c93-ca894a5ac169 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:43.678962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffdeb69-b6c4-4a4e-9c93-ca894a5ac169 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:43.687325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-fffdeb69-b6c4-4a4e-9c93-ca894a5ac169 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:43.687325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:43.687325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:43.687325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:43.687325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:43.687325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:43.687325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:43.687325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:43.687325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:43.687325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:43.687325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:43.687325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:43.687325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:43.687325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:43.687325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:43.687325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:43.688405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:43.688405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:43.688405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:43.688405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:43.688405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:43.688405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:43.688405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:43.688405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:43.688405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:43.688405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:43.688405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:43.688405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:43.688405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:43.688405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:43.688405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:43.688405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:43.689358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:43.689358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:43.689358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:43.689358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:43.689358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:43.689358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:43.689358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:43.689358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:43.689358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:43.689358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:43.689358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:43.689358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:43.691374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 190/384] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:43.692454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffdeb69-b6c4-4a4e-9c93-ca894a5ac169 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:43.724972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf977d76-3678-4247-aab3-1365791a2441 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:43.758784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6e0e6-c564-418b-9665-89366a9ac83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:43.760941 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6e0e6-c564-418b-9665-89366a9ac83a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:43.760941 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6e0e6-c564-418b-9665-89366a9ac83a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:43.760941 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6e0e6-c564-418b-9665-89366a9ac83a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:43.760941 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6e0e6-c564-418b-9665-89366a9ac83a None admin] RBAC: Adding query filter params (group.id=7527d489bc5f4d629f998c4b9f2d8d4b, role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=f7ec186b03924ec6a68eab5b0ea7d82d) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:43.761257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6e0e6-c564-418b-9665-89366a9ac83a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:43.763068 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b6e0e6-c564-418b-9665-89366a9ac83a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:43.770199 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 195/385] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jul 21 05:30:43 2022] GET /identity/v3/role_assignments?group.id=7527d489bc5f4d629f998c4b9f2d8d4b&scope.project.id=f7ec186b03924ec6a68eab5b0ea7d82d&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:43.771190 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6e0e6-c564-418b-9665-89366a9ac83a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:43.796026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:43.809605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffdeb69-b6c4-4a4e-9c93-ca894a5ac169 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:43.824349 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:43.850412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840207aa-5738-4116-bd14-7ceac1a12add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:43.851744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840207aa-5738-4116-bd14-7ceac1a12add None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:43.852238 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840207aa-5738-4116-bd14-7ceac1a12add None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:43.852736 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840207aa-5738-4116-bd14-7ceac1a12add None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:43.867782 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840207aa-5738-4116-bd14-7ceac1a12add None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:43.877031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-840207aa-5738-4116-bd14-7ceac1a12add None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:43.877031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:43.877031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:43.877031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:43.877031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:43.877031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:43.877031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:43.877031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:43.877031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:43.877031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:43.877031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:43.877031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:43.877031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:43.877031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:43.877031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:43.877031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:43.878025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:43.878025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:43.878025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:43.878025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:43.878025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:43.878025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:43.878025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:43.878025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:43.878025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:43.878025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:43.878025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:43.878025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:43.878025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:43.878025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:43.878025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:43.878025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:43.878823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:43.878823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:43.878823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:43.878823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:43.878823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:43.878823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:43.878823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:43.878823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:43.878823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:43.878823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:43.878823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:43.878823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:43.881678 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 191/386] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:30:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:43.892180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840207aa-5738-4116-bd14-7ceac1a12add None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:43.911006 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6e0e6-c564-418b-9665-89366a9ac83a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:44.010003 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95123b13-c39f-4ee6-bba3-a53abf299443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:44.011364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95123b13-c39f-4ee6-bba3-a53abf299443 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:44.011862 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95123b13-c39f-4ee6-bba3-a53abf299443 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:44.012598 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95123b13-c39f-4ee6-bba3-a53abf299443 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:44.014048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95123b13-c39f-4ee6-bba3-a53abf299443 None admin] RBAC: Adding query filter params (name=member) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:44.014746 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95123b13-c39f-4ee6-bba3-a53abf299443 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:44.016976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95123b13-c39f-4ee6-bba3-a53abf299443 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:44.028189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 196/387] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:44.033439 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95123b13-c39f-4ee6-bba3-a53abf299443 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:44.038240 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840207aa-5738-4116-bd14-7ceac1a12add None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:44.087569 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e81f7b-353b-4b8b-814b-576baa5a4f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:44.090138 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e81f7b-353b-4b8b-814b-576baa5a4f75 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:44.090912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e81f7b-353b-4b8b-814b-576baa5a4f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:44.091587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e81f7b-353b-4b8b-814b-576baa5a4f75 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:44.095430 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e81f7b-353b-4b8b-814b-576baa5a4f75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:44.095956 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e81f7b-353b-4b8b-814b-576baa5a4f75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:44.099868 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e81f7b-353b-4b8b-814b-576baa5a4f75 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:44.114156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 192/388] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:30:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:44.119914 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e81f7b-353b-4b8b-814b-576baa5a4f75 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:44.147477 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:44.193167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95123b13-c39f-4ee6-bba3-a53abf299443 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:44.194853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:44.248719 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e81f7b-353b-4b8b-814b-576baa5a4f75 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:44.284665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b274b3-b4fc-40da-a0a9-3ea94ca056cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:44.285268 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816a4c9-434f-4c2f-b1ed-110df68e7264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:44.286131 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b274b3-b4fc-40da-a0a9-3ea94ca056cd None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:44.286248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816a4c9-434f-4c2f-b1ed-110df68e7264 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:44.286425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816a4c9-434f-4c2f-b1ed-110df68e7264 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:44.286703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816a4c9-434f-4c2f-b1ed-110df68e7264 None admin] PATH_INFO: `/v3/groups/082eef03ba5f4329ab39e13dc4e287e2` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:44.287073 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b274b3-b4fc-40da-a0a9-3ea94ca056cd None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:44.287515 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b274b3-b4fc-40da-a0a9-3ea94ca056cd None admin] PATH_INFO: `/v3/groups/082eef03ba5f4329ab39e13dc4e287e2` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:44.288295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816a4c9-434f-4c2f-b1ed-110df68e7264 None admin] RBAC: Authorizing `identity:get_group(group_id=082eef03ba5f4329ab39e13dc4e287e2)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:44.289550 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b274b3-b4fc-40da-a0a9-3ea94ca056cd None admin] RBAC: Authorizing `identity:get_group(group_id=082eef03ba5f4329ab39e13dc4e287e2)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:44.289851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816a4c9-434f-4c2f-b1ed-110df68e7264 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:44.291424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b274b3-b4fc-40da-a0a9-3ea94ca056cd None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:44.293587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 193/389] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jul 21 05:30:44 2022] GET /identity/v3/groups/082eef03ba5f4329ab39e13dc4e287e2 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:44.295803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 197/390] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jul 21 05:30:44 2022] GET /identity/v3/groups/082eef03ba5f4329ab39e13dc4e287e2 => generated 226 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:44.302233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816a4c9-434f-4c2f-b1ed-110df68e7264 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:44.302516 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b274b3-b4fc-40da-a0a9-3ea94ca056cd None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:44.456892 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:44.488151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816a4c9-434f-4c2f-b1ed-110df68e7264 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:44.505168 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b274b3-b4fc-40da-a0a9-3ea94ca056cd None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:44.529037 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0f047-df54-4166-9a46-b862d87b23cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:44.529920 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0f047-df54-4166-9a46-b862d87b23cf None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:44.530204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0f047-df54-4166-9a46-b862d87b23cf None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:44.530481 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0f047-df54-4166-9a46-b862d87b23cf None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:44.532406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0f047-df54-4166-9a46-b862d87b23cf None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:44.533883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0f047-df54-4166-9a46-b862d87b23cf None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:44.537414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 198/391] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:44 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:44.542918 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0f047-df54-4166-9a46-b862d87b23cf None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:44.582842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa601ad4-317e-417b-9168-7338a49f6975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:44.583869 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa601ad4-317e-417b-9168-7338a49f6975 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:44.584156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa601ad4-317e-417b-9168-7338a49f6975 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:44.585382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa601ad4-317e-417b-9168-7338a49f6975 None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:44.585901 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa601ad4-317e-417b-9168-7338a49f6975 None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:44.587594 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa601ad4-317e-417b-9168-7338a49f6975 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:44.589374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 194/392] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:44 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:44.604359 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa601ad4-317e-417b-9168-7338a49f6975 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:44.627075 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:44.769587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0f047-df54-4166-9a46-b862d87b23cf None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:44.826219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa601ad4-317e-417b-9168-7338a49f6975 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:44.840708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b0462f-2005-4c64-9d57-d8950ecb3b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:44.841930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0462f-2005-4c64-9d57-d8950ecb3b28 None admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:44.842433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0462f-2005-4c64-9d57-d8950ecb3b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:44.842870 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0462f-2005-4c64-9d57-d8950ecb3b28 None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b/groups/082eef03ba5f4329ab39e13dc4e287e2/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:44.849722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b0462f-2005-4c64-9d57-d8950ecb3b28 None admin] RBAC: Authorizing `identity:create_grant(project_id=db74f3f0803a45359025fb277815314b, group_id=082eef03ba5f4329ab39e13dc4e287e2, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:44.851222 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b0462f-2005-4c64-9d57-d8950ecb3b28 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:44.878971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754febc-0a99-46fb-bfa7-46892bf26573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:44.889285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 199/393] 10.222.0.44 () {58 vars in 1442 bytes} [Thu Jul 21 05:30:44 2022] PUT /identity/v3/projects/db74f3f0803a45359025fb277815314b/groups/082eef03ba5f4329ab39e13dc4e287e2/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:30:44.889442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754febc-0a99-46fb-bfa7-46892bf26573 None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:44.889723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754febc-0a99-46fb-bfa7-46892bf26573 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:44.890064 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754febc-0a99-46fb-bfa7-46892bf26573 None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b/groups/082eef03ba5f4329ab39e13dc4e287e2/roles/a43ba93f358b4b1a8a445851a5a216a0` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:44.893360 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754febc-0a99-46fb-bfa7-46892bf26573 None admin] RBAC: Authorizing `identity:create_grant(project_id=db74f3f0803a45359025fb277815314b, group_id=082eef03ba5f4329ab39e13dc4e287e2, role_id=a43ba93f358b4b1a8a445851a5a216a0)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:44.894927 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754febc-0a99-46fb-bfa7-46892bf26573 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:44.925786 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 195/394] 10.222.0.44 () {58 vars in 1442 bytes} [Thu Jul 21 05:30:44 2022] PUT /identity/v3/projects/db74f3f0803a45359025fb277815314b/groups/082eef03ba5f4329ab39e13dc4e287e2/roles/a43ba93f358b4b1a8a445851a5a216a0 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:30:45.019712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967d498-673d-43d9-ada8-7a5c4bcf229e None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:45.020235 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967d498-673d-43d9-ada8-7a5c4bcf229e None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:45.020346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967d498-673d-43d9-ada8-7a5c4bcf229e None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:45.021855 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 200/395] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:45.028297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5c649-63ae-4fdf-8290-8e2aa0374d95 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:45.028891 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5c649-63ae-4fdf-8290-8e2aa0374d95 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:45.029270 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5c649-63ae-4fdf-8290-8e2aa0374d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:45.058310 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-21c5c649-63ae-4fdf-8290-8e2aa0374d95 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:45.058310 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-21c5c649-63ae-4fdf-8290-8e2aa0374d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:45.060083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:45.190624 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 196/396] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:45.219554 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967d498-673d-43d9-ada8-7a5c4bcf229e None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:45.350902 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:45.356072 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967d498-673d-43d9-ada8-7a5c4bcf229e None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:45.392909 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b022d13c-254f-4008-b51b-d1b451fc70c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:45.393725 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022d13c-254f-4008-b51b-d1b451fc70c7 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:45.394007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022d13c-254f-4008-b51b-d1b451fc70c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:45.394284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b022d13c-254f-4008-b51b-d1b451fc70c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:45.405682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b022d13c-254f-4008-b51b-d1b451fc70c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:45.414486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-b022d13c-254f-4008-b51b-d1b451fc70c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:45.414486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:45.414486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:45.414486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:45.414486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:45.414486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:45.414486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:45.414486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:45.414486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:45.414486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:45.414486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:45.414486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:45.414486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:45.414486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:45.414486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:45.414486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:45.415403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:45.415403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:45.415403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:45.415403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:45.415403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:45.415403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:45.415403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:45.415403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:45.415403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:45.415403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:45.415403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:45.415403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:45.415403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:45.415403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:45.415403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:45.415403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:45.416216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:45.416216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:45.416216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:45.416216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:45.416216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:45.416216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:45.416216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:45.416216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:45.416216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:45.416216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:45.416216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:45.416216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:45.416810 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 201/397] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:45.422306 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5c649-63ae-4fdf-8290-8e2aa0374d95 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:45.570046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5c649-63ae-4fdf-8290-8e2aa0374d95 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:45.618479 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691db0d-98e8-482b-b380-2b1f84d7f2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:45.619194 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691db0d-98e8-482b-b380-2b1f84d7f2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:45.619481 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691db0d-98e8-482b-b380-2b1f84d7f2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:45.619760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691db0d-98e8-482b-b380-2b1f84d7f2d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:45.620528 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691db0d-98e8-482b-b380-2b1f84d7f2d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:45.620832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691db0d-98e8-482b-b380-2b1f84d7f2d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:45.622229 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8691db0d-98e8-482b-b380-2b1f84d7f2d2 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:45.627846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:45.636657 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 197/398] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:45.642868 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691db0d-98e8-482b-b380-2b1f84d7f2d2 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:45.643399 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5f84f-3b2f-485e-813c-e718443f0014 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:45.643695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5f84f-3b2f-485e-813c-e718443f0014 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:45.643971 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5f84f-3b2f-485e-813c-e718443f0014 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:45.645585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 202/399] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:45.652041 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b344325-9b4f-43ec-9f92-492e327201f3 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:45.652041 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b344325-9b4f-43ec-9f92-492e327201f3 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:45.652041 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b344325-9b4f-43ec-9f92-492e327201f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:45.671802 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-1b344325-9b4f-43ec-9f92-492e327201f3 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:45.672994 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-1b344325-9b4f-43ec-9f92-492e327201f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:45.776178 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 203/400] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:45.813896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691db0d-98e8-482b-b380-2b1f84d7f2d2 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:45.820928 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b344325-9b4f-43ec-9f92-492e327201f3 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:45.886809 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae63b70-68ba-4fce-bc90-0413c6cabdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:45.887777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae63b70-68ba-4fce-bc90-0413c6cabdaf None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:45.888064 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae63b70-68ba-4fce-bc90-0413c6cabdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:45.888338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae63b70-68ba-4fce-bc90-0413c6cabdaf None admin] PATH_INFO: `/v3/groups/082eef03ba5f4329ab39e13dc4e287e2` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:45.889551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae63b70-68ba-4fce-bc90-0413c6cabdaf None admin] RBAC: Authorizing `identity:get_group(group_id=082eef03ba5f4329ab39e13dc4e287e2)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:45.891042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae63b70-68ba-4fce-bc90-0413c6cabdaf None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:45.893090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 198/401] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jul 21 05:30:45 2022] GET /identity/v3/groups/082eef03ba5f4329ab39e13dc4e287e2 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:45.900576 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae63b70-68ba-4fce-bc90-0413c6cabdaf None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:45.992708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:46.007527 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:46.014801 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae63b70-68ba-4fce-bc90-0413c6cabdaf None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:46.017906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b344325-9b4f-43ec-9f92-492e327201f3 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:46.058147 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee222dc-35a9-4797-afd9-ed7093d32827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:46.059041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee222dc-35a9-4797-afd9-ed7093d32827 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:46.059587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee222dc-35a9-4797-afd9-ed7093d32827 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:46.059587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee222dc-35a9-4797-afd9-ed7093d32827 None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:46.060885 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee222dc-35a9-4797-afd9-ed7093d32827 None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:46.066161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee222dc-35a9-4797-afd9-ed7093d32827 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:46.068938 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 199/402] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:45 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:46.075226 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee222dc-35a9-4797-afd9-ed7093d32827 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:46.082169 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18515f77-6771-4e72-a0df-2b0846cc162f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:46.084275 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18515f77-6771-4e72-a0df-2b0846cc162f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:46.084557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18515f77-6771-4e72-a0df-2b0846cc162f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:46.084829 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18515f77-6771-4e72-a0df-2b0846cc162f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:46.103935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18515f77-6771-4e72-a0df-2b0846cc162f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:46.114677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-18515f77-6771-4e72-a0df-2b0846cc162f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:46.114677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:46.114677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:46.114677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:46.114677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:46.114677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:46.114677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:46.114677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:46.114677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:46.114677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:46.114677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:46.114677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:46.114677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:46.114677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:46.114677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:46.114677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:46.115638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:46.115638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:46.115638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:46.115638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:46.115638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:46.115638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:46.115638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:46.115638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:46.115638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:46.115638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:46.115638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:46.115638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:46.115638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:46.115638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:46.115638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:46.115638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:46.116425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:46.116425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:46.116425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:46.116425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:46.116425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:46.116425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:46.116425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:46.116425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:46.116425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:46.116425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:46.116425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:46.116425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:46.117114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 204/403] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:30:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:30:46.122551 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18515f77-6771-4e72-a0df-2b0846cc162f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:46.225475 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee222dc-35a9-4797-afd9-ed7093d32827 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:46.261154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc911709-2b0f-4ea9-8747-bd763bb699b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:46.262396 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc911709-2b0f-4ea9-8747-bd763bb699b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:46.262688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc911709-2b0f-4ea9-8747-bd763bb699b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:46.262966 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc911709-2b0f-4ea9-8747-bd763bb699b9 None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b/groups/082eef03ba5f4329ab39e13dc4e287e2/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:46.265245 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc911709-2b0f-4ea9-8747-bd763bb699b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8855f4be124f44818348e5b04172899b, group_id=082eef03ba5f4329ab39e13dc4e287e2, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:46.266700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc911709-2b0f-4ea9-8747-bd763bb699b9 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:46.270035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:46.288250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 200/404] 10.222.0.44 () {58 vars in 1442 bytes} [Thu Jul 21 05:30:46 2022] PUT /identity/v3/projects/8855f4be124f44818348e5b04172899b/groups/082eef03ba5f4329ab39e13dc4e287e2/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:30:46.289686 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2650200-91ac-490a-a903-07083632a451 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:46.289981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2650200-91ac-490a-a903-07083632a451 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:46.290262 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2650200-91ac-490a-a903-07083632a451 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:46.292033 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 201/405] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:46.298069 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc31069-3fe5-444f-b4e4-0771e507ccef None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:46.298410 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc31069-3fe5-444f-b4e4-0771e507ccef None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:46.298625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc31069-3fe5-444f-b4e4-0771e507ccef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:46.317526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-8cc31069-3fe5-444f-b4e4-0771e507ccef None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:46.318577 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8cc31069-3fe5-444f-b4e4-0771e507ccef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:46.321370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18515f77-6771-4e72-a0df-2b0846cc162f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:46.367923 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2869c1-f9eb-4313-b7ee-01a010024a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:46.368973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2869c1-f9eb-4313-b7ee-01a010024a27 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:46.369905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2869c1-f9eb-4313-b7ee-01a010024a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:46.370379 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2869c1-f9eb-4313-b7ee-01a010024a27 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:46.371558 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2869c1-f9eb-4313-b7ee-01a010024a27 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:46.371874 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2869c1-f9eb-4313-b7ee-01a010024a27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:46.373316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2869c1-f9eb-4313-b7ee-01a010024a27 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:46.383445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 205/406] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:30:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:46.395654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2869c1-f9eb-4313-b7ee-01a010024a27 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:46.408941 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 202/407] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:46.420689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:46.438492 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc31069-3fe5-444f-b4e4-0771e507ccef None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:46.558953 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2869c1-f9eb-4313-b7ee-01a010024a27 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:46.565442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc31069-3fe5-444f-b4e4-0771e507ccef None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:46.598865 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:46.599832 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847f845-7205-4fba-97ce-c92a72921b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:46.600767 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847f845-7205-4fba-97ce-c92a72921b51 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:46.601065 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847f845-7205-4fba-97ce-c92a72921b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:46.601493 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847f845-7205-4fba-97ce-c92a72921b51 None admin] PATH_INFO: `/v3/groups/082eef03ba5f4329ab39e13dc4e287e2` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:46.602863 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847f845-7205-4fba-97ce-c92a72921b51 None admin] RBAC: Authorizing `identity:get_group(group_id=082eef03ba5f4329ab39e13dc4e287e2)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:46.605238 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847f845-7205-4fba-97ce-c92a72921b51 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:46.607358 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 206/408] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jul 21 05:30:46 2022] GET /identity/v3/groups/082eef03ba5f4329ab39e13dc4e287e2 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:46.615115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847f845-7205-4fba-97ce-c92a72921b51 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:46.619860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08590eb-5862-466d-9dc0-b604d676cae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:46.620689 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08590eb-5862-466d-9dc0-b604d676cae1 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:46.620977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08590eb-5862-466d-9dc0-b604d676cae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:46.621253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08590eb-5862-466d-9dc0-b604d676cae1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:46.649179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08590eb-5862-466d-9dc0-b604d676cae1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:46.656962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-f08590eb-5862-466d-9dc0-b604d676cae1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:46.656962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:46.656962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:46.656962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:46.656962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:46.656962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:46.656962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:46.656962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:46.656962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:46.656962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:46.656962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:46.656962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:46.656962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:46.656962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:46.656962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:46.656962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:46.657859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:46.657859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:46.657859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:46.657859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:46.657859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:46.657859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:46.657859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:46.657859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:46.657859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:46.657859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:46.657859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:46.657859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:46.657859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:46.657859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:46.657859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:46.657859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:46.658645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:46.658645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:46.658645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:46.658645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:46.658645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:46.658645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:46.658645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:46.658645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:46.658645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:46.658645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:46.658645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:46.658645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:46.668338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 203/409] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:30:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:46.688996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08590eb-5862-466d-9dc0-b604d676cae1 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:46.755404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:46.771945 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847f845-7205-4fba-97ce-c92a72921b51 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:46.809285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d8c25-0bb7-4fe3-b52b-d95b42eafa7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:46.810511 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d8c25-0bb7-4fe3-b52b-d95b42eafa7b None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:46.811034 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d8c25-0bb7-4fe3-b52b-d95b42eafa7b None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:46.811737 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d8c25-0bb7-4fe3-b52b-d95b42eafa7b None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:46.813260 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d8c25-0bb7-4fe3-b52b-d95b42eafa7b None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:46.815235 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d8c25-0bb7-4fe3-b52b-d95b42eafa7b None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:46.817455 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 207/410] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:46 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:46.823445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d8c25-0bb7-4fe3-b52b-d95b42eafa7b None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:46.935889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08590eb-5862-466d-9dc0-b604d676cae1 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:46.975277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d8c25-0bb7-4fe3-b52b-d95b42eafa7b None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:47.002567 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64102a3b-1e15-4e2c-b776-667d3cf1a58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:47.003829 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64102a3b-1e15-4e2c-b776-667d3cf1a58b None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:47.004111 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64102a3b-1e15-4e2c-b776-667d3cf1a58b None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:47.004390 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64102a3b-1e15-4e2c-b776-667d3cf1a58b None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:47.005155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64102a3b-1e15-4e2c-b776-667d3cf1a58b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:47.006265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64102a3b-1e15-4e2c-b776-667d3cf1a58b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:47.007694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64102a3b-1e15-4e2c-b776-667d3cf1a58b None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:47.026971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 204/411] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:47.032718 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64102a3b-1e15-4e2c-b776-667d3cf1a58b None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:47.047296 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fcd18-f2f8-4d55-adbc-4679b61acd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:47.048530 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fcd18-f2f8-4d55-adbc-4679b61acd45 None admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:47.049026 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fcd18-f2f8-4d55-adbc-4679b61acd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:47.049484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fcd18-f2f8-4d55-adbc-4679b61acd45 None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b/groups/082eef03ba5f4329ab39e13dc4e287e2/roles/a43ba93f358b4b1a8a445851a5a216a0` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:47.053723 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2fcd18-f2f8-4d55-adbc-4679b61acd45 None admin] RBAC: Authorizing `identity:create_grant(project_id=8855f4be124f44818348e5b04172899b, group_id=082eef03ba5f4329ab39e13dc4e287e2, role_id=a43ba93f358b4b1a8a445851a5a216a0)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:47.055510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2fcd18-f2f8-4d55-adbc-4679b61acd45 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:47.079664 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 208/412] 10.222.0.44 () {58 vars in 1442 bytes} [Thu Jul 21 05:30:46 2022] PUT /identity/v3/projects/8855f4be124f44818348e5b04172899b/groups/082eef03ba5f4329ab39e13dc4e287e2/roles/a43ba93f358b4b1a8a445851a5a216a0 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:30:47.125317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:47.178118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64102a3b-1e15-4e2c-b776-667d3cf1a58b None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:47.215532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ccb92-797f-4930-a348-5356728d4fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:47.216774 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ccb92-797f-4930-a348-5356728d4fce None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:47.217250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ccb92-797f-4930-a348-5356728d4fce None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:47.217712 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ccb92-797f-4930-a348-5356728d4fce None admin] PATH_INFO: `/v3/groups/7527d489bc5f4d629f998c4b9f2d8d4b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:47.219384 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ccb92-797f-4930-a348-5356728d4fce None admin] RBAC: Authorizing `identity:get_group(group_id=7527d489bc5f4d629f998c4b9f2d8d4b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:47.221080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ccb92-797f-4930-a348-5356728d4fce None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:47.223539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 205/413] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jul 21 05:30:47 2022] GET /identity/v3/groups/7527d489bc5f4d629f998c4b9f2d8d4b => generated 229 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:47.229662 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fcd18-f2f8-4d55-adbc-4679b61acd45 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:47.243410 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:47.341211 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fcd18-f2f8-4d55-adbc-4679b61acd45 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:47.374706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860c25f-9453-4d78-a813-7bb940746fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:47.375697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860c25f-9453-4d78-a813-7bb940746fae None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:47.375981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860c25f-9453-4d78-a813-7bb940746fae None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:47.376241 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860c25f-9453-4d78-a813-7bb940746fae None admin] PATH_INFO: `/v3/projects/f7ec186b03924ec6a68eab5b0ea7d82d` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:47.377480 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3860c25f-9453-4d78-a813-7bb940746fae None admin] RBAC: Authorizing `identity:get_project(project_id=f7ec186b03924ec6a68eab5b0ea7d82d)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:47.378982 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3860c25f-9453-4d78-a813-7bb940746fae None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:47.381058 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 209/414] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:47 2022] GET /identity/v3/projects/f7ec186b03924ec6a68eab5b0ea7d82d => generated 344 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:47.386872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ccb92-797f-4930-a348-5356728d4fce None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:47.477597 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd3232-bbe2-4116-93bc-6381e5fa8c8d None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:47.477786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd3232-bbe2-4116-93bc-6381e5fa8c8d None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:47.478058 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd3232-bbe2-4116-93bc-6381e5fa8c8d None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:47.479728 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 210/415] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:30:47.487371 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5aedfb-36b5-4a5e-b7e3-13656fb3bb54 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:47.487779 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5aedfb-36b5-4a5e-b7e3-13656fb3bb54 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:47.488485 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5aedfb-36b5-4a5e-b7e3-13656fb3bb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:47.509265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ccb92-797f-4930-a348-5356728d4fce None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:47.522095 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-8e5aedfb-36b5-4a5e-b7e3-13656fb3bb54 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:47.525964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8e5aedfb-36b5-4a5e-b7e3-13656fb3bb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:47.542060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:47.578772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34806ff7-d2e2-4019-b80e-f4962d5a3782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:47.580002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34806ff7-d2e2-4019-b80e-f4962d5a3782 None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:47.580457 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34806ff7-d2e2-4019-b80e-f4962d5a3782 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:47.580874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34806ff7-d2e2-4019-b80e-f4962d5a3782 None admin] PATH_INFO: `/v3/projects/f7ec186b03924ec6a68eab5b0ea7d82d/groups/7527d489bc5f4d629f998c4b9f2d8d4b/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:47.584848 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34806ff7-d2e2-4019-b80e-f4962d5a3782 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7ec186b03924ec6a68eab5b0ea7d82d, group_id=7527d489bc5f4d629f998c4b9f2d8d4b, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:47.587191 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34806ff7-d2e2-4019-b80e-f4962d5a3782 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:47.619188 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 206/416] 10.222.0.44 () {58 vars in 1442 bytes} [Thu Jul 21 05:30:47 2022] PUT /identity/v3/projects/f7ec186b03924ec6a68eab5b0ea7d82d/groups/7527d489bc5f4d629f998c4b9f2d8d4b/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:30:47.664744 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 211/417] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:47.681063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89b0d5-3cc2-45df-a351-276ed6e778c7 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:47.681398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89b0d5-3cc2-45df-a351-276ed6e778c7 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:47.681657 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89b0d5-3cc2-45df-a351-276ed6e778c7 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:47.683500 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 207/418] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:47.693664 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f9e52-771c-4818-8630-e6e5738f00e9 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:47.694285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f9e52-771c-4818-8630-e6e5738f00e9 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:47.694748 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f9e52-771c-4818-8630-e6e5738f00e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:47.700850 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89b0d5-3cc2-45df-a351-276ed6e778c7 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:47.721687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:47.732657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-495f9e52-771c-4818-8630-e6e5738f00e9 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:47.735533 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-495f9e52-771c-4818-8630-e6e5738f00e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:47.844106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 212/419] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:47.869522 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f9e52-771c-4818-8630-e6e5738f00e9 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:47.871211 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89b0d5-3cc2-45df-a351-276ed6e778c7 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:47.910397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb1a71-3d87-4bcc-bb96-4598ce70c209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:47.911287 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb1a71-3d87-4bcc-bb96-4598ce70c209 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:47.911565 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb1a71-3d87-4bcc-bb96-4598ce70c209 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:47.911836 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb1a71-3d87-4bcc-bb96-4598ce70c209 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:47.923923 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb1a71-3d87-4bcc-bb96-4598ce70c209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:47.932846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-4beb1a71-3d87-4bcc-bb96-4598ce70c209 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:47.932846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:47.932846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:47.932846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:47.932846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:47.932846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:47.932846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:47.932846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:47.932846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:47.932846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:47.932846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:47.932846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:47.932846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:47.932846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:47.932846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:47.932846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:47.933754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:47.933754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:47.933754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:47.933754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:47.933754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:47.933754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:47.933754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:47.933754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:47.933754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:47.933754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:47.933754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:47.933754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:47.933754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:47.933754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:47.933754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:47.933754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:47.934547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:47.934547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:47.934547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:47.934547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:47.934547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:47.934547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:47.934547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:47.934547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:47.934547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:47.934547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:47.934547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:47.934547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:47.934547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 208/420] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:47.942433 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb1a71-3d87-4bcc-bb96-4598ce70c209 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:48.022869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f9e52-771c-4818-8630-e6e5738f00e9 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:48.025501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:48.065768 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecceb703-4804-4b63-91f8-bd9fc2a37361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:48.066605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecceb703-4804-4b63-91f8-bd9fc2a37361 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:48.066883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecceb703-4804-4b63-91f8-bd9fc2a37361 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:48.067196 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecceb703-4804-4b63-91f8-bd9fc2a37361 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:48.072909 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:48.092322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecceb703-4804-4b63-91f8-bd9fc2a37361 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:48.110608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-ecceb703-4804-4b63-91f8-bd9fc2a37361 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:48.110608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:48.110608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:48.110608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:48.110608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:48.110608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:48.110608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:48.110608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:48.110608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:48.110608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:48.110608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:48.110608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:48.110608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:48.110608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:48.110608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:48.110608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:48.111617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:48.111617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:48.111617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:48.111617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:48.111617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:48.111617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:48.111617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:48.111617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:48.111617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:48.111617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:48.111617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:48.111617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:48.111617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:48.111617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:48.111617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:48.111617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:48.112422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:48.112422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:48.112422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:48.112422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:48.112422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:48.112422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:48.112422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:48.112422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:48.112422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:48.112422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:48.112422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:48.112422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:48.112422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 213/421] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:30:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:48.122973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecceb703-4804-4b63-91f8-bd9fc2a37361 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:48.165308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb1a71-3d87-4bcc-bb96-4598ce70c209 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:48.230718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecceb703-4804-4b63-91f8-bd9fc2a37361 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:48.262369 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beace35d-6122-4866-ac61-94d548f14393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:48.263060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beace35d-6122-4866-ac61-94d548f14393 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:48.263343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beace35d-6122-4866-ac61-94d548f14393 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:48.263619 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beace35d-6122-4866-ac61-94d548f14393 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:48.264366 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beace35d-6122-4866-ac61-94d548f14393 None admin] RBAC: Adding query filter params (name=member) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:48.264667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beace35d-6122-4866-ac61-94d548f14393 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:48.266050 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beace35d-6122-4866-ac61-94d548f14393 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:48.283108 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 209/422] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:48.283708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ae16b-8547-4dfe-9fff-b569ad004d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:48.284432 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ae16b-8547-4dfe-9fff-b569ad004d76 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:48.284718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ae16b-8547-4dfe-9fff-b569ad004d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:48.284994 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ae16b-8547-4dfe-9fff-b569ad004d76 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:48.285746 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ae16b-8547-4dfe-9fff-b569ad004d76 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:48.286049 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ae16b-8547-4dfe-9fff-b569ad004d76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:48.287510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43ae16b-8547-4dfe-9fff-b569ad004d76 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:48.290307 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beace35d-6122-4866-ac61-94d548f14393 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:48.309492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 214/423] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:30:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:48.315959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ae16b-8547-4dfe-9fff-b569ad004d76 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:48.406221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:48.416865 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beace35d-6122-4866-ac61-94d548f14393 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:48.437303 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:48.484786 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc968172-2d02-401d-8aee-533e902e7af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:48.485437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc968172-2d02-401d-8aee-533e902e7af5 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:48.485715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc968172-2d02-401d-8aee-533e902e7af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:48.485986 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc968172-2d02-401d-8aee-533e902e7af5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:48.511907 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ae16b-8547-4dfe-9fff-b569ad004d76 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:48.563311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b6d21-fbe1-493d-986c-8b7d9168a199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:48.563901 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b6d21-fbe1-493d-986c-8b7d9168a199 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:48.564189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b6d21-fbe1-493d-986c-8b7d9168a199 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:48.564474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b6d21-fbe1-493d-986c-8b7d9168a199 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:48.632296 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b6d21-fbe1-493d-986c-8b7d9168a199 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:48.633820 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b6d21-fbe1-493d-986c-8b7d9168a199 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:48.683917 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc968172-2d02-401d-8aee-533e902e7af5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:48.685404 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc968172-2d02-401d-8aee-533e902e7af5 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:48.710300 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:48.760194 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 215/424] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:48.761622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17c88e-a43b-467e-b5d9-4220db70297c None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:48.761963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17c88e-a43b-467e-b5d9-4220db70297c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:48.762249 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17c88e-a43b-467e-b5d9-4220db70297c None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:48.767732 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 216/425] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:48.767732 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17c88e-a43b-467e-b5d9-4220db70297c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:48.808255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 210/426] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:48.809979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39f831-9b85-4147-b1db-27cab0d0e995 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:48.810505 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39f831-9b85-4147-b1db-27cab0d0e995 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:48.810988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39f831-9b85-4147-b1db-27cab0d0e995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:48.827303 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:48.830335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-ef39f831-9b85-4147-b1db-27cab0d0e995 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:48.831879 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ef39f831-9b85-4147-b1db-27cab0d0e995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:48.867681 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17c88e-a43b-467e-b5d9-4220db70297c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:48.902477 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce415d-5593-4d5a-ac85-642d62868291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:48.903402 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce415d-5593-4d5a-ac85-642d62868291 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:48.903690 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce415d-5593-4d5a-ac85-642d62868291 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:48.903964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce415d-5593-4d5a-ac85-642d62868291 None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:48.905176 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce415d-5593-4d5a-ac85-642d62868291 None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:48.906642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce415d-5593-4d5a-ac85-642d62868291 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:48.908812 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 217/427] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:48 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:48.909691 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce415d-5593-4d5a-ac85-642d62868291 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:48.910262 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 211/428] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:48.920319 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39f831-9b85-4147-b1db-27cab0d0e995 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:48.950562 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:49.028235 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce415d-5593-4d5a-ac85-642d62868291 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:49.032271 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39f831-9b85-4147-b1db-27cab0d0e995 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:49.062313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:49.063996 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076bd0d5-fd59-4d1f-8e75-e0c6237ca8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:49.069277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bd0d5-fd59-4d1f-8e75-e0c6237ca8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:49.069587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bd0d5-fd59-4d1f-8e75-e0c6237ca8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:49.069876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bd0d5-fd59-4d1f-8e75-e0c6237ca8d0 None admin] PATH_INFO: `/v3/projects/db74f3f0803a45359025fb277815314b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:49.071586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076bd0d5-fd59-4d1f-8e75-e0c6237ca8d0 None admin] RBAC: Authorizing `identity:get_project(project_id=db74f3f0803a45359025fb277815314b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:49.073392 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076bd0d5-fd59-4d1f-8e75-e0c6237ca8d0 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:49.075358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa0cdb-06b9-4dc6-8b95-31e4dd7323f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:49.075650 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 218/429] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:48 2022] GET /identity/v3/projects/db74f3f0803a45359025fb277815314b => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:49.076456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076bd0d5-fd59-4d1f-8e75-e0c6237ca8d0 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:49.076963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa0cdb-06b9-4dc6-8b95-31e4dd7323f1 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:49.077525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa0cdb-06b9-4dc6-8b95-31e4dd7323f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:49.078054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa0cdb-06b9-4dc6-8b95-31e4dd7323f1 None admin] PATH_INFO: `/v3/groups/082eef03ba5f4329ab39e13dc4e287e2` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:49.079767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa0cdb-06b9-4dc6-8b95-31e4dd7323f1 None admin] RBAC: Authorizing `identity:get_group(group_id=082eef03ba5f4329ab39e13dc4e287e2)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:49.081616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa0cdb-06b9-4dc6-8b95-31e4dd7323f1 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:49.083803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 212/430] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jul 21 05:30:48 2022] GET /identity/v3/groups/082eef03ba5f4329ab39e13dc4e287e2 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:49.084806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa0cdb-06b9-4dc6-8b95-31e4dd7323f1 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:49.168857 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:49.177351 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076bd0d5-fd59-4d1f-8e75-e0c6237ca8d0 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:49.201355 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa0cdb-06b9-4dc6-8b95-31e4dd7323f1 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:49.211842 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47b212-2ed1-40c4-98c7-d039cfc2e1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:49.212653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47b212-2ed1-40c4-98c7-d039cfc2e1bc None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:49.212939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47b212-2ed1-40c4-98c7-d039cfc2e1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:49.213215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47b212-2ed1-40c4-98c7-d039cfc2e1bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:49.223580 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47b212-2ed1-40c4-98c7-d039cfc2e1bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:49.231322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-2c47b212-2ed1-40c4-98c7-d039cfc2e1bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:49.231322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:49.231322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:49.231322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:49.231322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:49.231322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:49.231322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:49.231322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:49.231322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:49.231322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:49.231322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:49.231322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:49.231322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:49.231322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:49.231322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:49.231322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:49.232613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:49.232613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:49.232613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:49.232613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:49.232613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:49.232613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:49.232613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:49.232613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:49.232613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:49.232613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:49.232613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:49.232613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:49.232613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:49.232613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:49.232613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:49.232613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:49.233545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:49.233545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:49.233545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:49.233545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:49.233545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:49.233545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:49.233545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:49.233545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:49.233545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:49.233545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:49.233545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 21 05:30:49.233545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:49.233545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 219/431] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:49.238105 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47b212-2ed1-40c4-98c7-d039cfc2e1bc None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:49.249496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7f34c-4d9e-4862-baab-45d35c3b52ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:49.250383 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7f34c-4d9e-4862-baab-45d35c3b52ac None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:49.250672 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7f34c-4d9e-4862-baab-45d35c3b52ac None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:49.251345 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f7f34c-4d9e-4862-baab-45d35c3b52ac None admin] PATH_INFO: `/v3/groups/082eef03ba5f4329ab39e13dc4e287e2` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:49.252543 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7f34c-4d9e-4862-baab-45d35c3b52ac None admin] RBAC: Authorizing `identity:get_group(group_id=082eef03ba5f4329ab39e13dc4e287e2)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:49.254641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f7f34c-4d9e-4862-baab-45d35c3b52ac None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:49.257159 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 213/432] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jul 21 05:30:49 2022] GET /identity/v3/groups/082eef03ba5f4329ab39e13dc4e287e2 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:49.258215 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7f34c-4d9e-4862-baab-45d35c3b52ac None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:49.331549 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:49.351648 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47b212-2ed1-40c4-98c7-d039cfc2e1bc None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:49.374956 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f7f34c-4d9e-4862-baab-45d35c3b52ac None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:49.386341 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b8bb0-1272-4382-b386-82dc33367ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:49.387114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b8bb0-1272-4382-b386-82dc33367ced None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:49.387406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b8bb0-1272-4382-b386-82dc33367ced None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:49.387689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b8bb0-1272-4382-b386-82dc33367ced None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:49.388589 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b8bb0-1272-4382-b386-82dc33367ced None admin] RBAC: Adding query filter params (group.id=082eef03ba5f4329ab39e13dc4e287e2, role.id=a43ba93f358b4b1a8a445851a5a216a0, scope.project.id=db74f3f0803a45359025fb277815314b) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:49.388910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b8bb0-1272-4382-b386-82dc33367ced None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:49.390367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b8bb0-1272-4382-b386-82dc33367ced None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:49.397500 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 220/433] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jul 21 05:30:49 2022] GET /identity/v3/role_assignments?group.id=082eef03ba5f4329ab39e13dc4e287e2&scope.project.id=db74f3f0803a45359025fb277815314b&role.id=a43ba93f358b4b1a8a445851a5a216a0 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:49.398336 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b8bb0-1272-4382-b386-82dc33367ced None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:49.424540 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:49.430159 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a726d-c19c-4dfe-a8cd-538b36a17149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:49.431308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a726d-c19c-4dfe-a8cd-538b36a17149 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:49.431799 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a726d-c19c-4dfe-a8cd-538b36a17149 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:49.432395 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a726d-c19c-4dfe-a8cd-538b36a17149 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:49.433379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a726d-c19c-4dfe-a8cd-538b36a17149 None admin] RBAC: Adding query filter params (name=member) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:49.433897 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a726d-c19c-4dfe-a8cd-538b36a17149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:49.443053 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a726d-c19c-4dfe-a8cd-538b36a17149 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:49.473482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 214/434] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:30:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:49.484551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a726d-c19c-4dfe-a8cd-538b36a17149 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:49.520020 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b8bb0-1272-4382-b386-82dc33367ced None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:49.562460 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab58a8-f8c6-4385-97c0-6e8c1b92031f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:49.563648 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab58a8-f8c6-4385-97c0-6e8c1b92031f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:49.564137 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab58a8-f8c6-4385-97c0-6e8c1b92031f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:49.564680 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab58a8-f8c6-4385-97c0-6e8c1b92031f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:49.566485 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab58a8-f8c6-4385-97c0-6e8c1b92031f None admin] RBAC: Adding query filter params (group.id=082eef03ba5f4329ab39e13dc4e287e2, role.id=b59c5b350c854281838a22884fd8f32f, scope.project.id=db74f3f0803a45359025fb277815314b) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:49.567085 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab58a8-f8c6-4385-97c0-6e8c1b92031f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:49.569864 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab58a8-f8c6-4385-97c0-6e8c1b92031f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:49.579483 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 221/435] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jul 21 05:30:49 2022] GET /identity/v3/role_assignments?group.id=082eef03ba5f4329ab39e13dc4e287e2&scope.project.id=db74f3f0803a45359025fb277815314b&role.id=b59c5b350c854281838a22884fd8f32f => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:49.582152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7a185-604a-4732-8202-4bc999dfe038 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:49.583038 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7a185-604a-4732-8202-4bc999dfe038 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:49.583532 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7a185-604a-4732-8202-4bc999dfe038 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:49.590708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 222/436] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:49.593171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf3435-40c1-4c79-9aef-5f8cd01808b4 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:49.593777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf3435-40c1-4c79-9aef-5f8cd01808b4 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:49.594266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf3435-40c1-4c79-9aef-5f8cd01808b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:49.615232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-3dbf3435-40c1-4c79-9aef-5f8cd01808b4 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:49.617561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3dbf3435-40c1-4c79-9aef-5f8cd01808b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:49.669032 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a726d-c19c-4dfe-a8cd-538b36a17149 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:49.698399 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:49.698593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 223/437] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:49.710632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ead43e-cc9e-4185-ab48-5eddafcbdbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:49.711671 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ead43e-cc9e-4185-ab48-5eddafcbdbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:49.712100 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ead43e-cc9e-4185-ab48-5eddafcbdbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:49.712542 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ead43e-cc9e-4185-ab48-5eddafcbdbd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:49.725601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf3435-40c1-4c79-9aef-5f8cd01808b4 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:49.739059 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:49.789143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ead43e-cc9e-4185-ab48-5eddafcbdbd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:49.790701 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ead43e-cc9e-4185-ab48-5eddafcbdbd8 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:49.842566 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf3435-40c1-4c79-9aef-5f8cd01808b4 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:49.881748 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db79e8-68d6-4b44-8bc8-6d5d02e893a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:49.883190 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db79e8-68d6-4b44-8bc8-6d5d02e893a3 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:49.883701 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db79e8-68d6-4b44-8bc8-6d5d02e893a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:49.884170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db79e8-68d6-4b44-8bc8-6d5d02e893a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:49.896605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db79e8-68d6-4b44-8bc8-6d5d02e893a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:49.905693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-06db79e8-68d6-4b44-8bc8-6d5d02e893a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:49.905693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:49.905693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:49.905693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:49.905693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:49.905693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:49.905693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:49.905693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:49.905693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:49.905693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:49.905693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:49.905693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:49.905693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:49.905693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:49.905693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:49.905693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:49.906621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:49.906621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:49.906621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:49.906621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:49.906621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:49.906621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:49.906621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:49.906621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:49.906621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:49.906621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:49.906621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:49.906621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:49.906621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:49.906621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:49.906621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:49.906621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:49.907584 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:49.907584 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:49.907584 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:49.907584 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:49.907584 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:49.907584 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:49.907584 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:49.907584 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:49.907584 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:49.907584 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:49.907584 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:49.907584 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:49.913364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 215/438] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 05:30:49.913451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 224/439] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:30:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:30:49.915306 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ead43e-cc9e-4185-ab48-5eddafcbdbd8 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:49.920002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db79e8-68d6-4b44-8bc8-6d5d02e893a3 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:49.945125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:50.009662 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:50.036934 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ead43e-cc9e-4185-ab48-5eddafcbdbd8 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:50.046481 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db79e8-68d6-4b44-8bc8-6d5d02e893a3 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:50.077825 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8672e-be66-4714-94a8-a9a45fa4a5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:50.078692 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8672e-be66-4714-94a8-a9a45fa4a5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:50.078997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8672e-be66-4714-94a8-a9a45fa4a5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:50.079265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8672e-be66-4714-94a8-a9a45fa4a5a1 None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:50.080495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba8672e-be66-4714-94a8-a9a45fa4a5a1 None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:50.081968 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba8672e-be66-4714-94a8-a9a45fa4a5a1 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:50.082335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c4ee3-1057-4ed8-80fa-db4ac43db416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:50.083248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c4ee3-1057-4ed8-80fa-db4ac43db416 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:50.083713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 216/440] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:49 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:50.083871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c4ee3-1057-4ed8-80fa-db4ac43db416 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:50.084363 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c4ee3-1057-4ed8-80fa-db4ac43db416 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:50.084883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b21f0-e479-44ac-afce-df939bacfe9c None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:50.085165 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b21f0-e479-44ac-afce-df939bacfe9c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:50.085440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b21f0-e479-44ac-afce-df939bacfe9c None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:50.085976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c4ee3-1057-4ed8-80fa-db4ac43db416 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:50.086474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c4ee3-1057-4ed8-80fa-db4ac43db416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:50.087773 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 217/441] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:50.088241 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c4ee3-1057-4ed8-80fa-db4ac43db416 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:50.089156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b21f0-e479-44ac-afce-df939bacfe9c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:50.097683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 225/442] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:30:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:50.098903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c4f95-c357-4848-91ce-137427c2f635 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:50.099320 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c4f95-c357-4848-91ce-137427c2f635 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:50.099601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c4f95-c357-4848-91ce-137427c2f635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:50.117770 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-3b1c4f95-c357-4848-91ce-137427c2f635 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:50.119007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3b1c4f95-c357-4848-91ce-137427c2f635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:50.195541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 226/443] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:50.196113 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c4f95-c357-4848-91ce-137427c2f635 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:50.198525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:50.206768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b21f0-e479-44ac-afce-df939bacfe9c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:50.244598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d66ae-62a8-46fa-b902-85fea918c72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:50.245470 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d66ae-62a8-46fa-b902-85fea918c72c None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:50.245748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d66ae-62a8-46fa-b902-85fea918c72c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:50.246498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d66ae-62a8-46fa-b902-85fea918c72c None admin] PATH_INFO: `/v3/groups/082eef03ba5f4329ab39e13dc4e287e2` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:50.247365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d66ae-62a8-46fa-b902-85fea918c72c None admin] RBAC: Authorizing `identity:get_group(group_id=082eef03ba5f4329ab39e13dc4e287e2)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:50.248720 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d66ae-62a8-46fa-b902-85fea918c72c None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:50.250626 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 218/444] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jul 21 05:30:50 2022] GET /identity/v3/groups/082eef03ba5f4329ab39e13dc4e287e2 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:50.251503 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d66ae-62a8-46fa-b902-85fea918c72c None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:50.266922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:50.304501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c4f95-c357-4848-91ce-137427c2f635 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:50.338621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc642c22-25b4-4eb8-ac51-767c249e222a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:50.339314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc642c22-25b4-4eb8-ac51-767c249e222a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:50.339596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc642c22-25b4-4eb8-ac51-767c249e222a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:50.339867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc642c22-25b4-4eb8-ac51-767c249e222a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:50.349242 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d66ae-62a8-46fa-b902-85fea918c72c None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:50.383194 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b96df-76ff-45bf-9038-38a04e3fe54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:50.384098 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b96df-76ff-45bf-9038-38a04e3fe54c None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:50.384382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b96df-76ff-45bf-9038-38a04e3fe54c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:50.384653 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b96df-76ff-45bf-9038-38a04e3fe54c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:50.395166 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b96df-76ff-45bf-9038-38a04e3fe54c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:50.403530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-721b96df-76ff-45bf-9038-38a04e3fe54c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:50.403530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:50.403530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:50.403530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:50.403530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:50.403530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:50.403530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:50.403530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:50.403530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:50.403530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:50.403530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:50.403530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:50.403530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:50.403530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:50.403530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:50.403530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:50.404581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:50.404581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:50.404581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:50.404581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:50.404581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:50.404581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:50.404581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:50.404581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:50.404581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:50.404581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:50.404581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:50.404581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:50.404581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:50.404581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:50.404581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:50.404581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:50.405376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:50.405376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:50.405376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:50.405376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:50.405376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:50.405376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:50.405376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:50.405376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:50.405376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:50.405376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:50.405376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:30:50.405376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:50.405997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc642c22-25b4-4eb8-ac51-767c249e222a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:50.406069 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 219/445] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:30:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:50.406069 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b96df-76ff-45bf-9038-38a04e3fe54c None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:50.406185 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc642c22-25b4-4eb8-ac51-767c249e222a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:50.430625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:50.466059 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:50.503297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b96df-76ff-45bf-9038-38a04e3fe54c None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:50.504320 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 227/446] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:30:50.505133 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc642c22-25b4-4eb8-ac51-767c249e222a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:50.543643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9a52a-4779-43a5-80ab-a3c5ac3dca42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:50.544302 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9a52a-4779-43a5-80ab-a3c5ac3dca42 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:50.544585 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9a52a-4779-43a5-80ab-a3c5ac3dca42 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:50.544858 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9a52a-4779-43a5-80ab-a3c5ac3dca42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:50.545689 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9a52a-4779-43a5-80ab-a3c5ac3dca42 None admin] RBAC: Adding query filter params (group.id=082eef03ba5f4329ab39e13dc4e287e2, role.id=b59c5b350c854281838a22884fd8f32f, scope.project.id=8855f4be124f44818348e5b04172899b) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:50.545995 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9a52a-4779-43a5-80ab-a3c5ac3dca42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:50.547506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9a52a-4779-43a5-80ab-a3c5ac3dca42 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:50.553357 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 220/447] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jul 21 05:30:50 2022] GET /identity/v3/role_assignments?group.id=082eef03ba5f4329ab39e13dc4e287e2&scope.project.id=8855f4be124f44818348e5b04172899b&role.id=b59c5b350c854281838a22884fd8f32f => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:50.554143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9a52a-4779-43a5-80ab-a3c5ac3dca42 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:50.608278 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc642c22-25b4-4eb8-ac51-767c249e222a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:50.649773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af56885-9de1-4c53-b592-956a52702c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:50.650972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af56885-9de1-4c53-b592-956a52702c5a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:50.651525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af56885-9de1-4c53-b592-956a52702c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:50.652010 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af56885-9de1-4c53-b592-956a52702c5a None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:50.653409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af56885-9de1-4c53-b592-956a52702c5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:50.653976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af56885-9de1-4c53-b592-956a52702c5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:50.656134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af56885-9de1-4c53-b592-956a52702c5a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:50.666797 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 228/448] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:50.670888 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:50.672536 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af56885-9de1-4c53-b592-956a52702c5a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:50.688879 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9a52a-4779-43a5-80ab-a3c5ac3dca42 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:50.699764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:50.732892 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d482f58-fa22-40fc-b863-4b7cbbd3b030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:50.733861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d482f58-fa22-40fc-b863-4b7cbbd3b030 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:50.734145 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d482f58-fa22-40fc-b863-4b7cbbd3b030 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:50.734415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d482f58-fa22-40fc-b863-4b7cbbd3b030 None admin] PATH_INFO: `/v3/projects/8855f4be124f44818348e5b04172899b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:50.735603 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d482f58-fa22-40fc-b863-4b7cbbd3b030 None admin] RBAC: Authorizing `identity:get_project(project_id=8855f4be124f44818348e5b04172899b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:50.737063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d482f58-fa22-40fc-b863-4b7cbbd3b030 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:50.738764 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 221/449] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:50 2022] GET /identity/v3/projects/8855f4be124f44818348e5b04172899b => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:50.744330 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d482f58-fa22-40fc-b863-4b7cbbd3b030 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:50.796267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af56885-9de1-4c53-b592-956a52702c5a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:50.836823 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43896f8e-b880-49c2-adac-316b7808cd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:50.837556 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43896f8e-b880-49c2-adac-316b7808cd87 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:50.837846 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43896f8e-b880-49c2-adac-316b7808cd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:50.838123 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43896f8e-b880-49c2-adac-316b7808cd87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:50.878650 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d482f58-fa22-40fc-b863-4b7cbbd3b030 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:50.928058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25af548-ca38-4c1e-81ff-d6427c87f39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:50.928942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25af548-ca38-4c1e-81ff-d6427c87f39d None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:50.929205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25af548-ca38-4c1e-81ff-d6427c87f39d None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:50.929480 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25af548-ca38-4c1e-81ff-d6427c87f39d None admin] PATH_INFO: `/v3/groups/082eef03ba5f4329ab39e13dc4e287e2` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:50.932089 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25af548-ca38-4c1e-81ff-d6427c87f39d None admin] RBAC: Authorizing `identity:get_group(group_id=082eef03ba5f4329ab39e13dc4e287e2)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:50.933549 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25af548-ca38-4c1e-81ff-d6427c87f39d None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:50.940153 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 222/450] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jul 21 05:30:50 2022] GET /identity/v3/groups/082eef03ba5f4329ab39e13dc4e287e2 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:50.940908 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25af548-ca38-4c1e-81ff-d6427c87f39d None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:50.941968 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43896f8e-b880-49c2-adac-316b7808cd87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:50.943621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43896f8e-b880-49c2-adac-316b7808cd87 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:50.967512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:50.968707 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:51.050148 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 229/451] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 05:30:51.051730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25af548-ca38-4c1e-81ff-d6427c87f39d None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:51.055672 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43896f8e-b880-49c2-adac-316b7808cd87 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:51.085827 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229fec7-d193-401c-912e-9a1b5256ea90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:51.086528 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229fec7-d193-401c-912e-9a1b5256ea90 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:51.086828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229fec7-d193-401c-912e-9a1b5256ea90 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:51.087137 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229fec7-d193-401c-912e-9a1b5256ea90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:51.087943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229fec7-d193-401c-912e-9a1b5256ea90 None admin] RBAC: Adding query filter params (group.id=082eef03ba5f4329ab39e13dc4e287e2, role.id=a43ba93f358b4b1a8a445851a5a216a0, scope.project.id=8855f4be124f44818348e5b04172899b) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:51.088258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229fec7-d193-401c-912e-9a1b5256ea90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:51.089767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229fec7-d193-401c-912e-9a1b5256ea90 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:51.095864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 223/452] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jul 21 05:30:50 2022] GET /identity/v3/role_assignments?group.id=082eef03ba5f4329ab39e13dc4e287e2&scope.project.id=8855f4be124f44818348e5b04172899b&role.id=a43ba93f358b4b1a8a445851a5a216a0 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:51.167672 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43896f8e-b880-49c2-adac-316b7808cd87 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:51.193811 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:51.206332 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84c087-3234-4c6e-aba0-bfab8cda2fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:51.207254 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84c087-3234-4c6e-aba0-bfab8cda2fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:51.207536 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84c087-3234-4c6e-aba0-bfab8cda2fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:51.207813 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84c087-3234-4c6e-aba0-bfab8cda2fb9 None admin] PATH_INFO: `/v3/projects/f7ec186b03924ec6a68eab5b0ea7d82d` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:51.209131 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84c087-3234-4c6e-aba0-bfab8cda2fb9 None admin] RBAC: Authorizing `identity:get_project(project_id=f7ec186b03924ec6a68eab5b0ea7d82d)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:51.210607 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84c087-3234-4c6e-aba0-bfab8cda2fb9 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:51.212406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 230/453] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:30:51 2022] GET /identity/v3/projects/f7ec186b03924ec6a68eab5b0ea7d82d => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:51.218510 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229fec7-d193-401c-912e-9a1b5256ea90 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:51.307479 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:51.324040 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229fec7-d193-401c-912e-9a1b5256ea90 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:51.358297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83a9a0-a251-478c-8fb1-dabe6fc3d19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:51.359219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83a9a0-a251-478c-8fb1-dabe6fc3d19b None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:51.359496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83a9a0-a251-478c-8fb1-dabe6fc3d19b None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:51.359779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83a9a0-a251-478c-8fb1-dabe6fc3d19b None admin] PATH_INFO: `/v3/groups/7527d489bc5f4d629f998c4b9f2d8d4b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:51.360958 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83a9a0-a251-478c-8fb1-dabe6fc3d19b None admin] RBAC: Authorizing `identity:get_group(group_id=7527d489bc5f4d629f998c4b9f2d8d4b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:51.362426 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83a9a0-a251-478c-8fb1-dabe6fc3d19b None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:51.364096 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 224/454] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jul 21 05:30:51 2022] GET /identity/v3/groups/7527d489bc5f4d629f998c4b9f2d8d4b => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:51.370249 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84c087-3234-4c6e-aba0-bfab8cda2fb9 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:51.471004 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84c087-3234-4c6e-aba0-bfab8cda2fb9 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:51.504367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc6e1d-6443-4791-9168-bd804ec8d0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:51.505020 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc6e1d-6443-4791-9168-bd804ec8d0fb None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:51.505305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc6e1d-6443-4791-9168-bd804ec8d0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:51.505582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc6e1d-6443-4791-9168-bd804ec8d0fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:51.506416 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc6e1d-6443-4791-9168-bd804ec8d0fb None admin] RBAC: Adding query filter params (group.id=7527d489bc5f4d629f998c4b9f2d8d4b, role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=f7ec186b03924ec6a68eab5b0ea7d82d) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:51.506726 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc6e1d-6443-4791-9168-bd804ec8d0fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:51.508260 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc6e1d-6443-4791-9168-bd804ec8d0fb None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:51.514161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 231/455] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jul 21 05:30:51 2022] GET /identity/v3/role_assignments?group.id=7527d489bc5f4d629f998c4b9f2d8d4b&scope.project.id=f7ec186b03924ec6a68eab5b0ea7d82d&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:54.096903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0ec89-d6b5-444e-92cf-5be421d4ee8f None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:54.097406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0ec89-d6b5-444e-92cf-5be421d4ee8f None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:54.097406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0ec89-d6b5-444e-92cf-5be421d4ee8f None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:54.098977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 225/456] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:54.106215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e690d-e880-46e4-a1a8-cd66026f54ac None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:54.106636 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e690d-e880-46e4-a1a8-cd66026f54ac None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:54.106807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e690d-e880-46e4-a1a8-cd66026f54ac None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:54.109443 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7d525-ec9a-4ff8-853e-082b298b7968 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:54.109728 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7d525-ec9a-4ff8-853e-082b298b7968 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:54.110000 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7d525-ec9a-4ff8-853e-082b298b7968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:54.111945 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 232/457] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:54.116149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2a29b-5677-49b2-8dd8-d9e352531805 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:54.116429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2a29b-5677-49b2-8dd8-d9e352531805 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:54.116703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2a29b-5677-49b2-8dd8-d9e352531805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:54.136921 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-19d7d525-ec9a-4ff8-853e-082b298b7968 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:54.138097 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-19d7d525-ec9a-4ff8-853e-082b298b7968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:54.150186 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-cfc2a29b-5677-49b2-8dd8-d9e352531805 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:54.151374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-cfc2a29b-5677-49b2-8dd8-d9e352531805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:54.234214 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:54.248635 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 226/458] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:54.250574 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee0a21-f325-4f0a-ac44-4d0e1d129cef None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:54.250977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee0a21-f325-4f0a-ac44-4d0e1d129cef None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:54.251293 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee0a21-f325-4f0a-ac44-4d0e1d129cef None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:54.252556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 227/459] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:54.254252 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 233/460] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:54.257904 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896e57a-2cd3-4894-8f5c-64a9e72f0dca None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:54.258187 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896e57a-2cd3-4894-8f5c-64a9e72f0dca None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:54.258462 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896e57a-2cd3-4894-8f5c-64a9e72f0dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:54.284015 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-5896e57a-2cd3-4894-8f5c-64a9e72f0dca None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:54.285058 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2a29b-5677-49b2-8dd8-d9e352531805 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:54.285215 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5896e57a-2cd3-4894-8f5c-64a9e72f0dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:54.377721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 228/461] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:54.379977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896e57a-2cd3-4894-8f5c-64a9e72f0dca None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:54.400129 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2a29b-5677-49b2-8dd8-d9e352531805 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:54.407743 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:54.441913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525d346-8568-450d-a781-0736a9b8e337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:54.442987 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525d346-8568-450d-a781-0736a9b8e337 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:54.443418 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525d346-8568-450d-a781-0736a9b8e337 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:54.443793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f525d346-8568-450d-a781-0736a9b8e337 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:54.456784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525d346-8568-450d-a781-0736a9b8e337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:54.458707 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f525d346-8568-450d-a781-0736a9b8e337 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:54.479285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-f525d346-8568-450d-a781-0736a9b8e337 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:30:54.479285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:54.479285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:54.479285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:54.479285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:54.479285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:54.479285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:54.479285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:54.479285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:54.479285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:54.479285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:54.479285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:54.479285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:30:54.479285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:30:54.479285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:30:54.480253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:30:54.480253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:54.480253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:54.480253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:54.480253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:54.480253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:54.480253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:54.480253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:54.480253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:54.480253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:54.480253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:54.480253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:54.480253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:54.480253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:54.480253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:54.480253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:54.480253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:54.481101 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:30:54.481101 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:30:54.481101 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:30:54.481101 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:54.481330 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 234/462] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:30:54.487398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525d346-8568-450d-a781-0736a9b8e337 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:54.502616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896e57a-2cd3-4894-8f5c-64a9e72f0dca None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:54.530325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:54.538181 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f87a7-93c1-455d-9d64-1e761c14dab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:54.539055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f87a7-93c1-455d-9d64-1e761c14dab3 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:54.539326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f87a7-93c1-455d-9d64-1e761c14dab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:54.539598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f87a7-93c1-455d-9d64-1e761c14dab3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:54.549927 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f87a7-93c1-455d-9d64-1e761c14dab3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:54.551415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f87a7-93c1-455d-9d64-1e761c14dab3 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:54.568970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-319f87a7-93c1-455d-9d64-1e761c14dab3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:30:54.568970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:54.568970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:54.568970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:54.568970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:54.568970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:54.568970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:54.568970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:54.568970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:54.568970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:54.568970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:54.568970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:54.568970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:30:54.568970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:30:54.568970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:30:54.569855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:30:54.569855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:54.569855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:54.569855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:54.569855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:54.569855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:54.569855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:54.569855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:54.569855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:54.569855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:54.569855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:54.569855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:54.569855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:54.569855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:54.569855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:54.569855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:54.569855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:54.570824 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:30:54.570824 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:30:54.570824 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:30:54.570824 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:54.570824 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 229/463] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:54.571545 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f87a7-93c1-455d-9d64-1e761c14dab3 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:54.602679 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f525d346-8568-450d-a781-0736a9b8e337 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:54.637133 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:54.640863 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37e6c5-5b59-422d-b164-12b9f471201e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:54.642335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37e6c5-5b59-422d-b164-12b9f471201e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:54.642335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37e6c5-5b59-422d-b164-12b9f471201e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:54.642335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37e6c5-5b59-422d-b164-12b9f471201e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:54.659036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37e6c5-5b59-422d-b164-12b9f471201e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:54.659036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37e6c5-5b59-422d-b164-12b9f471201e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:54.675628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f87a7-93c1-455d-9d64-1e761c14dab3 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:54.676299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-2b37e6c5-5b59-422d-b164-12b9f471201e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:30:54.676299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:54.676299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:54.676299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:54.676299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:54.676299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:54.676299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:54.676299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:54.676299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:54.676299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:54.676299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:54.676299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:54.676299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:30:54.676299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:30:54.676299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:30:54.677124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:30:54.677124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:54.677124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:54.677124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:54.677124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:54.677124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:54.677124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:54.677124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:54.677124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:54.677124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:54.677124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:54.677124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:54.677124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:54.677124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:54.677124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:54.677124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:54.677124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:54.677965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:30:54.677965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:30:54.677965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:30:54.677965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:54.677965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 235/464] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:54.683032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37e6c5-5b59-422d-b164-12b9f471201e None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:54.713616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41618a-f498-4499-bbf9-1cdc118e3a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:54.714286 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41618a-f498-4499-bbf9-1cdc118e3a56 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:54.714570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41618a-f498-4499-bbf9-1cdc118e3a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:54.714844 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41618a-f498-4499-bbf9-1cdc118e3a56 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:54.715627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41618a-f498-4499-bbf9-1cdc118e3a56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:54.715932 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41618a-f498-4499-bbf9-1cdc118e3a56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:54.717312 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41618a-f498-4499-bbf9-1cdc118e3a56 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:54.734925 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 230/465] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:54.735800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41618a-f498-4499-bbf9-1cdc118e3a56 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:54.782645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:54.793655 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37e6c5-5b59-422d-b164-12b9f471201e None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:54.830621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0092cbd-b43c-45a8-a73f-6e5b99cef882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:54.831341 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0092cbd-b43c-45a8-a73f-6e5b99cef882 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:54.831622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0092cbd-b43c-45a8-a73f-6e5b99cef882 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:54.831899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0092cbd-b43c-45a8-a73f-6e5b99cef882 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:54.832647 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0092cbd-b43c-45a8-a73f-6e5b99cef882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:54.832955 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0092cbd-b43c-45a8-a73f-6e5b99cef882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:54.834364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0092cbd-b43c-45a8-a73f-6e5b99cef882 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:54.839071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41618a-f498-4499-bbf9-1cdc118e3a56 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:54.845778 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 236/466] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:54.848463 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0092cbd-b43c-45a8-a73f-6e5b99cef882 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:54.869072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:54.879853 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941700e-9003-4c8f-b96f-2ffc5851e256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:54.880896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941700e-9003-4c8f-b96f-2ffc5851e256 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:54.881420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941700e-9003-4c8f-b96f-2ffc5851e256 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:54.881874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941700e-9003-4c8f-b96f-2ffc5851e256 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:54.882882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941700e-9003-4c8f-b96f-2ffc5851e256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:54.883509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941700e-9003-4c8f-b96f-2ffc5851e256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:54.885240 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941700e-9003-4c8f-b96f-2ffc5851e256 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:54.897929 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 231/467] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:54.899653 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941decc-e117-45eb-83df-93938dceccf5 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:54.899940 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941decc-e117-45eb-83df-93938dceccf5 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:54.900218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941decc-e117-45eb-83df-93938dceccf5 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:54.901603 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 232/468] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:54.902168 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941decc-e117-45eb-83df-93938dceccf5 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:54.953271 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0092cbd-b43c-45a8-a73f-6e5b99cef882 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:54.989316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d573303-7c1d-41a8-8321-768149830d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:54.990400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d573303-7c1d-41a8-8321-768149830d9d None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:54.990676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d573303-7c1d-41a8-8321-768149830d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:54.990960 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d573303-7c1d-41a8-8321-768149830d9d None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:54.992081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d573303-7c1d-41a8-8321-768149830d9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:54.993908 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d573303-7c1d-41a8-8321-768149830d9d None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:55.015717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 237/469] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:30:54 2022] POST /identity/v3/users => generated 248 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:30:55.017103 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590daed-ccbf-4e8c-ab03-fdeaf8a4fae9 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:55.017464 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590daed-ccbf-4e8c-ab03-fdeaf8a4fae9 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:55.017814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590daed-ccbf-4e8c-ab03-fdeaf8a4fae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:55.028999 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941decc-e117-45eb-83df-93938dceccf5 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:55.040737 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:55.049199 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-b590daed-ccbf-4e8c-ab03-fdeaf8a4fae9 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:55.050584 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b590daed-ccbf-4e8c-ab03-fdeaf8a4fae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:55.081692 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307e039-d720-449a-a2c3-19b28e1333d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:55.082812 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307e039-d720-449a-a2c3-19b28e1333d8 None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:55.083202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307e039-d720-449a-a2c3-19b28e1333d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:55.083566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307e039-d720-449a-a2c3-19b28e1333d8 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:55.084484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307e039-d720-449a-a2c3-19b28e1333d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:55.087148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4307e039-d720-449a-a2c3-19b28e1333d8 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:55.123239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 233/470] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:30:54 2022] POST /identity/v3/users => generated 246 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:30:55.126481 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307e039-d720-449a-a2c3-19b28e1333d8 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:55.155434 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 238/471] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:55.200736 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:55.205884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590daed-ccbf-4e8c-ab03-fdeaf8a4fae9 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:55.304055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307e039-d720-449a-a2c3-19b28e1333d8 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:55.331478 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590daed-ccbf-4e8c-ab03-fdeaf8a4fae9 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:55.353722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e120d4-497b-4214-962e-fbf2b2a6b266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:55.353722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e120d4-497b-4214-962e-fbf2b2a6b266 None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:55.354016 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e120d4-497b-4214-962e-fbf2b2a6b266 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:55.354274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e120d4-497b-4214-962e-fbf2b2a6b266 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:55.355170 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e120d4-497b-4214-962e-fbf2b2a6b266 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:55.356540 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e120d4-497b-4214-962e-fbf2b2a6b266 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:55.382177 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 234/472] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:30:55 2022] POST /identity/v3/users => generated 248 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:30:55.384089 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d1355-7090-4624-bb9e-592c0fadd25d None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:55.384510 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d1355-7090-4624-bb9e-592c0fadd25d None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:55.384919 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d1355-7090-4624-bb9e-592c0fadd25d None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:55.386694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 235/473] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:55.389766 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745dbb7-3ec7-4204-92e5-bb82fca606f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:55.391082 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745dbb7-3ec7-4204-92e5-bb82fca606f3 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:55.391591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745dbb7-3ec7-4204-92e5-bb82fca606f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:55.392115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745dbb7-3ec7-4204-92e5-bb82fca606f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:55.392818 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c8344-d61d-4551-a5f1-4abd80318627 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:55.393109 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c8344-d61d-4551-a5f1-4abd80318627 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:55.393420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c8344-d61d-4551-a5f1-4abd80318627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:55.397659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:55.410249 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745dbb7-3ec7-4204-92e5-bb82fca606f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:55.412444 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745dbb7-3ec7-4204-92e5-bb82fca606f3 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:55.413128 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-a65c8344-d61d-4551-a5f1-4abd80318627 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:55.414431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a65c8344-d61d-4551-a5f1-4abd80318627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:55.434783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-2745dbb7-3ec7-4204-92e5-bb82fca606f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:30:55.434783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:55.434783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:55.434783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:55.434783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:55.434783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:55.434783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:55.434783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:55.434783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:55.434783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:55.434783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:55.434783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:55.434783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:30:55.434783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:30:55.434783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:30:55.435792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:30:55.435792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:55.435792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:55.435792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:55.435792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:55.435792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:55.435792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:55.435792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:55.435792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:55.435792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:55.435792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:55.435792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:55.435792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:55.435792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:55.435792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:55.435792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:55.435792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:55.436642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:30:55.436642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:30:55.436642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:30:55.436642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:55.436642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 239/474] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:30:55.442119 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745dbb7-3ec7-4204-92e5-bb82fca606f3 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:55.502102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 236/475] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:55.526929 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c8344-d61d-4551-a5f1-4abd80318627 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:55.562408 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:55.582444 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745dbb7-3ec7-4204-92e5-bb82fca606f3 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:55.628857 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b7b5d-f71b-49c1-ae14-404f2047ccda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:55.630016 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b7b5d-f71b-49c1-ae14-404f2047ccda None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:55.630292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b7b5d-f71b-49c1-ae14-404f2047ccda None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:55.630566 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b7b5d-f71b-49c1-ae14-404f2047ccda None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:55.631738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b7b5d-f71b-49c1-ae14-404f2047ccda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:55.632326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b7b5d-f71b-49c1-ae14-404f2047ccda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:55.634072 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b7b5d-f71b-49c1-ae14-404f2047ccda None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:55.652887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 240/476] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:30:55.659153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b7b5d-f71b-49c1-ae14-404f2047ccda None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:55.668003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c8344-d61d-4551-a5f1-4abd80318627 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:55.712524 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c54cf-3c26-4d14-9e39-7408b0a77d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:55.713443 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c54cf-3c26-4d14-9e39-7408b0a77d74 None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:55.713733 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c54cf-3c26-4d14-9e39-7408b0a77d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:55.714333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c54cf-3c26-4d14-9e39-7408b0a77d74 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:55.715437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c54cf-3c26-4d14-9e39-7408b0a77d74 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:55.717043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c54cf-3c26-4d14-9e39-7408b0a77d74 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:55.722612 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:55.725527 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.sql.core [None req-823c54cf-3c26-4d14-9e39-7408b0a77d74 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 05:30:55.725527 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 05:30:55.725527 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [parameters: {'id': 'cde86154a793489187381f466b195c13', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 05:30:55.725527 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80390) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 05:30:55.750632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-823c54cf-3c26-4d14-9e39-7408b0a77d74 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 05:30:55.750632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:55.750632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 05:30:55.750632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 05:30:55.750632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 05:30:55.750632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 05:30:55.750632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 05:30:55.750632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 05:30:55.750632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 05:30:55.750632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application conn.query(q) Jul 21 05:30:55.750632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 05:30:55.750632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 05:30:55.750632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 05:30:55.750632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application result.read() Jul 21 05:30:55.750632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return ref.to_dict() Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 05:30:55.751605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application next(self.gen) Jul 21 05:30:55.752523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 05:30:55.752523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application yield resource Jul 21 05:30:55.752523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 05:30:55.752523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application next(self.gen) Jul 21 05:30:55.752523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 05:30:55.752523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 05:30:55.752523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 05:30:55.752523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 05:30:55.752523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 05:30:55.752523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise self.value Jul 21 05:30:55.752523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 05:30:55.752523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 05:30:55.752523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 05:30:55.752523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application session.commit() Jul 21 05:30:55.752523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 05:30:55.752523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 05:30:55.752523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 05:30:55.753397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 05:30:55.753397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 05:30:55.753397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self.session.flush() Jul 21 05:30:55.753397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 05:30:55.753397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 05:30:55.753397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 05:30:55.753397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 05:30:55.753397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 05:30:55.753397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application compat.raise_( Jul 21 05:30:55.753397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 05:30:55.753397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception Jul 21 05:30:55.753397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 05:30:55.753397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 05:30:55.753397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 05:30:55.753397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 05:30:55.753397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 05:30:55.753397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 05:30:55.755176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 05:30:55.755176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 05:30:55.755176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 05:30:55.755176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 05:30:55.755176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 05:30:55.755176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 05:30:55.755176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 05:30:55.755176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 05:30:55.755176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 05:30:55.755176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 05:30:55.755176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 05:30:55.755176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 05:30:55.755176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 05:30:55.755176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 05:30:55.755176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 05:30:55.755176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception Jul 21 05:30:55.756193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 05:30:55.756193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 05:30:55.756193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 05:30:55.756193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 05:30:55.756193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 05:30:55.756193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 05:30:55.756193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 05:30:55.756193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application conn.query(q) Jul 21 05:30:55.756193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 05:30:55.756193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 05:30:55.756193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 05:30:55.756193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application result.read() Jul 21 05:30:55.756193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 05:30:55.756193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 05:30:55.756193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 05:30:55.756193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 05:30:55.757193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 05:30:55.757193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 05:30:55.757193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 05:30:55.757193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 05:30:55.757193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 21 05:30:55.757193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 21 05:30:55.757193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [parameters: {'id': 'cde86154a793489187381f466b195c13', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 21 05:30:55.757193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 05:30:55.757193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:55.757193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:30:55.757193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:55.757193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:55.757193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:55.757193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:55.757193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:55.757193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:55.759435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:55.759435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:55.759435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:55.759435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:55.759435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:55.759435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:55.759435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 21 05:30:55.759435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 21 05:30:55.759435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:55.759435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:55.759435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 21 05:30:55.759435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 21 05:30:55.759435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 05:30:55.759435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 05:30:55.759435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 21 05:30:55.759435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:55.761350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 237/477] 10.222.0.44 () {60 vars in 1131 bytes} [Thu Jul 21 05:30:55 2022] POST /identity/v3/roles => generated 144 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:30:55.761350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c54cf-3c26-4d14-9e39-7408b0a77d74 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:55.787342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b7b5d-f71b-49c1-ae14-404f2047ccda None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:55.835215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76cb5da-7127-42ef-b999-9a7b0bdc27f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:55.836026 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cb5da-7127-42ef-b999-9a7b0bdc27f5 None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:55.836391 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cb5da-7127-42ef-b999-9a7b0bdc27f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:55.836664 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cb5da-7127-42ef-b999-9a7b0bdc27f5 None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:55.837394 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76cb5da-7127-42ef-b999-9a7b0bdc27f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:55.839168 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76cb5da-7127-42ef-b999-9a7b0bdc27f5 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:55.868001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 241/478] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:30:55 2022] POST /identity/v3/users => generated 249 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:30:55.875998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:55.886869 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c54cf-3c26-4d14-9e39-7408b0a77d74 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:55.926691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932ef8b-62cd-4fa2-8352-efe1eb66e4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:55.927972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932ef8b-62cd-4fa2-8352-efe1eb66e4ba None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:55.928442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932ef8b-62cd-4fa2-8352-efe1eb66e4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:55.928893 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932ef8b-62cd-4fa2-8352-efe1eb66e4ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:55.941907 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932ef8b-62cd-4fa2-8352-efe1eb66e4ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:55.950257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-7932ef8b-62cd-4fa2-8352-efe1eb66e4ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:55.950257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:55.950257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:55.950257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:55.950257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:55.950257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:55.950257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:55.950257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:55.950257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:55.950257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:55.950257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:55.950257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:55.950257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:55.950257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:55.950257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:55.950257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:55.952442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:55.952442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:55.952442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:55.952442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:55.952442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:55.952442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:55.952442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:55.952442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:55.952442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:55.952442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:55.952442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:55.952442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:55.952442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:55.952442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:55.952442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:55.952442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:55.954106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:55.954106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:55.954106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:55.954106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:55.954106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:55.954106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:55.954106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:55.954106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:55.954106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:55.954106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:55.954106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 21 05:30:55.954106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:55.954106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 238/479] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:30:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:55.958225 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76cb5da-7127-42ef-b999-9a7b0bdc27f5 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:56.067006 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76cb5da-7127-42ef-b999-9a7b0bdc27f5 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:56.069783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:56.102716 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970708e3-d4a4-4e17-ba0b-f79463968aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:56.103644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970708e3-d4a4-4e17-ba0b-f79463968aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:56.104243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970708e3-d4a4-4e17-ba0b-f79463968aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:56.104519 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970708e3-d4a4-4e17-ba0b-f79463968aa0 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:56.105275 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970708e3-d4a4-4e17-ba0b-f79463968aa0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:56.105581 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970708e3-d4a4-4e17-ba0b-f79463968aa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:56.106972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970708e3-d4a4-4e17-ba0b-f79463968aa0 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:56.115953 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 242/480] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:30:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:57.604037 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5304f-f78f-4303-bee7-a4223829652b None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:57.605086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5304f-f78f-4303-bee7-a4223829652b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:57.605636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5304f-f78f-4303-bee7-a4223829652b None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:57.607513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 239/481] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:57.612958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b91006-288b-4abc-8c2c-97446d5cb634 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:57.613790 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b91006-288b-4abc-8c2c-97446d5cb634 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:57.614266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b91006-288b-4abc-8c2c-97446d5cb634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:57.639333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-09b91006-288b-4abc-8c2c-97446d5cb634 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:57.640945 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-09b91006-288b-4abc-8c2c-97446d5cb634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:57.753056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 243/482] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:30:57.788614 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5304f-f78f-4303-bee7-a4223829652b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:57.838492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf0fd2-bd83-4c5d-9252-9c2a8f3a51af None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:57.838853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf0fd2-bd83-4c5d-9252-9c2a8f3a51af None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:57.839056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf0fd2-bd83-4c5d-9252-9c2a8f3a51af None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:57.840735 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 244/483] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:30:57.845585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9046018-8a51-4acd-a2c6-dddb52688703 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:57.845874 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9046018-8a51-4acd-a2c6-dddb52688703 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:57.846155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9046018-8a51-4acd-a2c6-dddb52688703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:57.864770 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-c9046018-8a51-4acd-a2c6-dddb52688703 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:57.865871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-c9046018-8a51-4acd-a2c6-dddb52688703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:57.966264 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5304f-f78f-4303-bee7-a4223829652b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:57.991123 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 245/484] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:58.023272 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb3426-226f-4fd5-a1e5-c2cad95c5fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:58.024087 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb3426-226f-4fd5-a1e5-c2cad95c5fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:58.024374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb3426-226f-4fd5-a1e5-c2cad95c5fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:58.024651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb3426-226f-4fd5-a1e5-c2cad95c5fd3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:58.031815 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9046018-8a51-4acd-a2c6-dddb52688703 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:58.050601 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb3426-226f-4fd5-a1e5-c2cad95c5fd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:58.058326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-6fbb3426-226f-4fd5-a1e5-c2cad95c5fd3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:30:58.058326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:58.058326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:58.058326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:58.058326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:58.058326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:58.058326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:58.058326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:58.058326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:58.058326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:58.058326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:58.058326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:58.058326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:58.058326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:58.058326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:58.058326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:58.059250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:58.059250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:58.059250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:58.059250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:58.059250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:58.059250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:58.059250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:58.059250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:58.059250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:58.059250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:58.059250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:58.059250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:58.059250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:58.059250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:58.059250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:58.059250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:58.060063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:58.060063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:58.060063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:58.060063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:58.060063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:58.060063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:58.060063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:58.060063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:58.060063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:58.060063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:58.060063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:30:58.060063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:58.060674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 240/485] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:30:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:58.069983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb3426-226f-4fd5-a1e5-c2cad95c5fd3 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:58.083394 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:58.106872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:58.166258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9046018-8a51-4acd-a2c6-dddb52688703 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:58.197302 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb3426-226f-4fd5-a1e5-c2cad95c5fd3 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:58.206172 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d60516-bbe2-413a-a400-06ecaf7c37d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:58.207169 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d60516-bbe2-413a-a400-06ecaf7c37d5 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:58.207553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d60516-bbe2-413a-a400-06ecaf7c37d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:58.207928 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d60516-bbe2-413a-a400-06ecaf7c37d5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:58.221609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d60516-bbe2-413a-a400-06ecaf7c37d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:58.232978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-06d60516-bbe2-413a-a400-06ecaf7c37d5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:30:58.232978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:58.232978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:58.232978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:58.232978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:58.232978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:58.232978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:58.232978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:58.232978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:58.232978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:58.232978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:58.232978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:58.232978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:58.232978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:58.232978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:58.232978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:58.234118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:58.234118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:58.234118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:58.234118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:58.234118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:58.234118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:58.234118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:58.234118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:58.234118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:58.234118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:58.234118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:58.234118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:58.234118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:58.234118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:58.234118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:58.234118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:58.234958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:58.234958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:58.234958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:58.234958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:58.234958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:58.234958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:58.234958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:58.234958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:58.234958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:58.234958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:58.234958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:30:58.234958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:58.239089 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 246/486] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:30:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:30:58.240653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51634723-9e2d-48a3-aeea-8b491e2447bf None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:58.240938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51634723-9e2d-48a3-aeea-8b491e2447bf None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:58.241220 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51634723-9e2d-48a3-aeea-8b491e2447bf None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:58.242801 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 247/487] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:58.243340 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51634723-9e2d-48a3-aeea-8b491e2447bf None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:58.246072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c1232-526d-45e7-9c6f-04756adba994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:58.247099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c1232-526d-45e7-9c6f-04756adba994 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:58.247582 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c1232-526d-45e7-9c6f-04756adba994 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:58.248069 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c1232-526d-45e7-9c6f-04756adba994 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:58.249045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c1232-526d-45e7-9c6f-04756adba994 None admin] RBAC: Adding query filter params (name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:58.249548 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c1232-526d-45e7-9c6f-04756adba994 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:58.251163 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7c1232-526d-45e7-9c6f-04756adba994 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:58.260687 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 241/488] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:30:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:58.262262 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006b72-ee2b-4c96-a057-f4db6e6b29e2 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:58.262767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006b72-ee2b-4c96-a057-f4db6e6b29e2 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:58.263339 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006b72-ee2b-4c96-a057-f4db6e6b29e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:58.280492 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-20006b72-ee2b-4c96-a057-f4db6e6b29e2 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:58.281972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-20006b72-ee2b-4c96-a057-f4db6e6b29e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:58.339408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:58.352299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51634723-9e2d-48a3-aeea-8b491e2447bf None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:58.360710 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 242/489] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:58.362364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20006b72-ee2b-4c96-a057-f4db6e6b29e2 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:58.366891 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:58.400952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf554731-d32f-4a8a-9da5-7b84a284871b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:58.402236 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf554731-d32f-4a8a-9da5-7b84a284871b None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:58.402899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf554731-d32f-4a8a-9da5-7b84a284871b None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:58.403375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf554731-d32f-4a8a-9da5-7b84a284871b None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:58.404898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf554731-d32f-4a8a-9da5-7b84a284871b None admin] RBAC: Adding query filter params (name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:58.405383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf554731-d32f-4a8a-9da5-7b84a284871b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:58.407842 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf554731-d32f-4a8a-9da5-7b84a284871b None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:58.418906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 248/490] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:30:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:58.420147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf554731-d32f-4a8a-9da5-7b84a284871b None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:58.492606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20006b72-ee2b-4c96-a057-f4db6e6b29e2 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:58.529963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b34c4-28ad-4301-a4b0-11108d2b0d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:58.530876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b34c4-28ad-4301-a4b0-11108d2b0d30 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:58.531421 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b34c4-28ad-4301-a4b0-11108d2b0d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:58.531771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b34c4-28ad-4301-a4b0-11108d2b0d30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:58.538621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf554731-d32f-4a8a-9da5-7b84a284871b None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:58.543479 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b34c4-28ad-4301-a4b0-11108d2b0d30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:58.545086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b34c4-28ad-4301-a4b0-11108d2b0d30 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:58.565688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-280b34c4-28ad-4301-a4b0-11108d2b0d30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:30:58.565688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:58.565688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:58.565688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:58.565688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:58.565688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:58.565688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:58.565688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:58.565688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:58.565688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:58.565688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:58.565688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:58.565688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:30:58.565688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:30:58.565688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:30:58.566632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:30:58.566632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:58.566632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:58.566632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:58.566632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:58.566632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:58.566632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:58.566632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:58.566632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:58.566632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:58.566632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:58.566632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:58.566632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:58.566632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:58.566632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:58.566632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:58.566632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:58.567516 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:30:58.567516 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:30:58.567516 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:30:58.567516 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:58.567516 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 243/491] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:58.568760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b34c4-28ad-4301-a4b0-11108d2b0d30 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:58.577044 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850eda6-9c42-416e-b9e5-dfef304d4963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:58.578194 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850eda6-9c42-416e-b9e5-dfef304d4963 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:58.578646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850eda6-9c42-416e-b9e5-dfef304d4963 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:58.579120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850eda6-9c42-416e-b9e5-dfef304d4963 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:58.590084 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9850eda6-9c42-416e-b9e5-dfef304d4963 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:58.598400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-9850eda6-9c42-416e-b9e5-dfef304d4963 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:30:58.598400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:58.598400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:58.598400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:58.598400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:58.598400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:58.598400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:58.598400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:58.598400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:58.598400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:58.598400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:58.598400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:58.598400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:58.598400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:58.598400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:58.598400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:58.599367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:58.599367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:58.599367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:58.599367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:58.599367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:58.599367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:58.599367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:58.599367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:58.599367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:58.599367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:58.599367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:58.599367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:58.599367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:58.599367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:58.599367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:58.599367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:58.600191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:58.600191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:58.600191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:58.600191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:58.600191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:58.600191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:58.600191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:58.600191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:58.600191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:58.600191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:58.600191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:30:58.600191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:58.604301 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 249/492] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:30:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:58.604301 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850eda6-9c42-416e-b9e5-dfef304d4963 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:58.616383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:58.629560 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:58.685416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b34c4-28ad-4301-a4b0-11108d2b0d30 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:58.714796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850eda6-9c42-416e-b9e5-dfef304d4963 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:58.718241 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94688a5-fe42-4328-b2c0-5c5a6943edbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:58.719090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94688a5-fe42-4328-b2c0-5c5a6943edbe None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:58.719342 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94688a5-fe42-4328-b2c0-5c5a6943edbe None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:58.719611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94688a5-fe42-4328-b2c0-5c5a6943edbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:58.729814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94688a5-fe42-4328-b2c0-5c5a6943edbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:58.731324 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94688a5-fe42-4328-b2c0-5c5a6943edbe None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:58.748397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-e94688a5-fe42-4328-b2c0-5c5a6943edbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:30:58.748397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:58.748397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:58.748397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:58.748397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:58.748397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:58.748397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:58.748397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:58.748397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:58.748397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:58.748397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:58.748397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:58.748397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:30:58.748397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:30:58.748397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:30:58.749398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:30:58.749398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:58.749398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:58.749398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:58.749398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:58.749398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:58.749398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:58.749398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:58.749398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:58.749398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:58.749398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:58.749398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:58.749398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:58.749398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:58.749398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:58.749398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:58.749398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:58.750255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:30:58.750255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:30:58.750255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:30:58.750255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:58.750466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca766b-92e9-474d-bdf4-cbb014e4e16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:58.750551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 244/493] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:58.750618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca766b-92e9-474d-bdf4-cbb014e4e16c None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:58.750618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca766b-92e9-474d-bdf4-cbb014e4e16c None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:58.750618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca766b-92e9-474d-bdf4-cbb014e4e16c None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:58.750796 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4578c-3d61-431c-971c-a1a915970c6c None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:58.751242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca766b-92e9-474d-bdf4-cbb014e4e16c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:58.751339 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4578c-3d61-431c-971c-a1a915970c6c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:58.751422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca766b-92e9-474d-bdf4-cbb014e4e16c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:58.751498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4578c-3d61-431c-971c-a1a915970c6c None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:58.755069 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca766b-92e9-474d-bdf4-cbb014e4e16c None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:58.755193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 245/494] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:58.755193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4578c-3d61-431c-971c-a1a915970c6c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:58.765983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 250/495] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:58.767441 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3672d-3052-472b-afb5-ec6d6ad642bd None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:58.767737 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3672d-3052-472b-afb5-ec6d6ad642bd None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:58.768010 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3672d-3052-472b-afb5-ec6d6ad642bd None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:58.771419 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 251/496] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:30:58.771419 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3672d-3052-472b-afb5-ec6d6ad642bd None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:58.835141 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:58.855632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4578c-3d61-431c-971c-a1a915970c6c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:58.864577 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:58.868474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3672d-3052-472b-afb5-ec6d6ad642bd None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:58.888661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d6de5-6706-4952-a97d-d479a265cf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:58.889333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d6de5-6706-4952-a97d-d479a265cf9f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:58.889616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d6de5-6706-4952-a97d-d479a265cf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:58.889895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d6de5-6706-4952-a97d-d479a265cf9f None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:58.890652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d6de5-6706-4952-a97d-d479a265cf9f None admin] RBAC: Adding query filter params (name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:58.890964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d6de5-6706-4952-a97d-d479a265cf9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:58.892391 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d6de5-6706-4952-a97d-d479a265cf9f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:58.900543 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 246/497] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:30:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:58.900770 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d379e8-3184-4a1e-913f-ee527314b9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:58.901434 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d379e8-3184-4a1e-913f-ee527314b9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:58.901706 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748221d6-d46f-4637-96db-99ff47e2c26b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:58.901821 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d379e8-3184-4a1e-913f-ee527314b9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:58.901995 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748221d6-d46f-4637-96db-99ff47e2c26b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:58.902089 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d379e8-3184-4a1e-913f-ee527314b9e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:58.902274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748221d6-d46f-4637-96db-99ff47e2c26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:58.902796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d379e8-3184-4a1e-913f-ee527314b9e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:58.903136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d379e8-3184-4a1e-913f-ee527314b9e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:58.904504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d379e8-3184-4a1e-913f-ee527314b9e8 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:58.916733 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 252/498] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:58.917525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d379e8-3184-4a1e-913f-ee527314b9e8 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:58.918384 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-748221d6-d46f-4637-96db-99ff47e2c26b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:58.919601 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-748221d6-d46f-4637-96db-99ff47e2c26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:58.986068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 247/499] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:58.988738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac0b88-1c15-47f6-87ff-3948dd6b8e58 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:58.989224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac0b88-1c15-47f6-87ff-3948dd6b8e58 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:58.989721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac0b88-1c15-47f6-87ff-3948dd6b8e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:59.009381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-f2ac0b88-1c15-47f6-87ff-3948dd6b8e58 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:30:59.010559 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f2ac0b88-1c15-47f6-87ff-3948dd6b8e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:30:59.019486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d379e8-3184-4a1e-913f-ee527314b9e8 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:59.044789 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:59.052630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbcd93-3596-42cc-a065-795e30768e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:59.053545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbcd93-3596-42cc-a065-795e30768e9d None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:59.053814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbcd93-3596-42cc-a065-795e30768e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:59.054106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cbcd93-3596-42cc-a065-795e30768e9d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:59.062480 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbcd93-3596-42cc-a065-795e30768e9d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:59.063986 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cbcd93-3596-42cc-a065-795e30768e9d None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:59.074901 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 248/500] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:30:59.075782 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac0b88-1c15-47f6-87ff-3948dd6b8e58 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:59.078784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-79cbcd93-3596-42cc-a065-795e30768e9d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:30:59.078784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:59.078784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:59.078784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:59.078784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:59.078784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:59.078784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:59.078784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:59.078784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:59.078784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:59.078784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:59.078784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:59.078784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:30:59.078784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:30:59.078784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:30:59.078784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:59.079895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:59.080814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:59.080814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:59.080814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:30:59.080814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:30:59.080814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:30:59.080814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:30:59.080814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:30:59.080814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:30:59.080814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:30:59.080814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:59.080814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 253/501] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jul 21 05:30:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:59.080814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbcd93-3596-42cc-a065-795e30768e9d None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:59.109045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:59.178542 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac0b88-1c15-47f6-87ff-3948dd6b8e58 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:59.183999 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cbcd93-3596-42cc-a065-795e30768e9d None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:59.212526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387832f-f48e-4212-8e6f-4922609edce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:59.213436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387832f-f48e-4212-8e6f-4922609edce8 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:59.213709 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387832f-f48e-4212-8e6f-4922609edce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:59.213979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387832f-f48e-4212-8e6f-4922609edce8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:59.216984 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da071c3-923e-4ffc-abb5-1a16b405c83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:59.217790 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da071c3-923e-4ffc-abb5-1a16b405c83c None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:59.218072 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da071c3-923e-4ffc-abb5-1a16b405c83c None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:59.218344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da071c3-923e-4ffc-abb5-1a16b405c83c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:59.223981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387832f-f48e-4212-8e6f-4922609edce8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:59.225452 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387832f-f48e-4212-8e6f-4922609edce8 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:59.226597 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da071c3-923e-4ffc-abb5-1a16b405c83c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:59.228062 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da071c3-923e-4ffc-abb5-1a16b405c83c None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:59.241686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-1da071c3-923e-4ffc-abb5-1a16b405c83c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 05:30:59.241686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:59.241686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:59.241686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:59.241686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:59.241686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:59.241686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:59.241686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:59.241686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:59.241686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:59.241686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:59.241686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:59.241686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:30:59.241686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:30:59.241686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:30:59.241686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:59.242669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:59.243595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:59.243595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:59.243595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:30:59.243595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:30:59.243595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:30:59.243595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:30:59.243595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:30:59.243595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:30:59.243595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 05:30:59.243595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:59.244102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-c387832f-f48e-4212-8e6f-4922609edce8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:30:59.244102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:59.244102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:59.244102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:59.244102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:59.244102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:59.244102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:59.244102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:59.244102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:59.244102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:59.244102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:59.244102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:59.244102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:30:59.244102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:30:59.244102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:30:59.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:30:59.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:59.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:59.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:59.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:59.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:59.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:59.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:59.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:59.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:59.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:59.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:59.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:59.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:59.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:59.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:59.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:59.246779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:30:59.246779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:30:59.246779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:30:59.246779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:30:59.247193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 254/502] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:30:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:59.247193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da071c3-923e-4ffc-abb5-1a16b405c83c None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:59.247411 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 249/503] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:59.247411 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387832f-f48e-4212-8e6f-4922609edce8 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:59.294615 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:59.323411 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:59.341195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387832f-f48e-4212-8e6f-4922609edce8 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:59.342601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da071c3-923e-4ffc-abb5-1a16b405c83c None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:59.374309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38535f1f-6f04-4dc8-8da7-1749f8d8e3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:59.374886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae224262-c83d-40c3-a243-79810e2ff3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:59.374994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38535f1f-6f04-4dc8-8da7-1749f8d8e3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:59.375267 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38535f1f-6f04-4dc8-8da7-1749f8d8e3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:59.375543 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38535f1f-6f04-4dc8-8da7-1749f8d8e3e4 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:59.375708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae224262-c83d-40c3-a243-79810e2ff3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:59.375989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae224262-c83d-40c3-a243-79810e2ff3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:59.376263 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae224262-c83d-40c3-a243-79810e2ff3d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:59.376381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38535f1f-6f04-4dc8-8da7-1749f8d8e3e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:59.376691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38535f1f-6f04-4dc8-8da7-1749f8d8e3e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:59.378122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38535f1f-6f04-4dc8-8da7-1749f8d8e3e4 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:59.386032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae224262-c83d-40c3-a243-79810e2ff3d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:30:59.393673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-ae224262-c83d-40c3-a243-79810e2ff3d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:30:59.393673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:59.393673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:59.393673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:59.393673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:59.393673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:59.393673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:59.393673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:59.393673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:59.393673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:59.393673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:59.393673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:59.393673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:30:59.393673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:30:59.393673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:30:59.393673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:30:59.394599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:30:59.394599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:30:59.394599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:30:59.394599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:30:59.394599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:59.394599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:59.394599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:59.394599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:59.394599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:59.394599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:59.394599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:59.394599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:59.394599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:59.394599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:59.394599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:59.394599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:59.395421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:59.395421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:59.395421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:59.395421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:59.395421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:30:59.395421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:30:59.395421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:30:59.395421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:30:59.395421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:30:59.395421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:30:59.395421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:30:59.395421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:59.395421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 255/504] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:30:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:59.396086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae224262-c83d-40c3-a243-79810e2ff3d6 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:59.419375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 250/505] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jul 21 05:30:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:59.420226 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38535f1f-6f04-4dc8-8da7-1749f8d8e3e4 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:59.501445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae224262-c83d-40c3-a243-79810e2ff3d6 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:59.510042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:59.522051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38535f1f-6f04-4dc8-8da7-1749f8d8e3e4 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:59.535162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a5331-aede-40eb-a2c6-7ad4bf70b8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:59.535980 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a5331-aede-40eb-a2c6-7ad4bf70b8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:59.536262 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a5331-aede-40eb-a2c6-7ad4bf70b8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:59.536533 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a5331-aede-40eb-a2c6-7ad4bf70b8b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:59.547856 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a5331-aede-40eb-a2c6-7ad4bf70b8b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:59.549327 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a5331-aede-40eb-a2c6-7ad4bf70b8b8 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:59.554986 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958ad8f-194f-42d4-ad15-3ececb2f0ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:59.555693 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958ad8f-194f-42d4-ad15-3ececb2f0ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:59.555975 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958ad8f-194f-42d4-ad15-3ececb2f0ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:59.556250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958ad8f-194f-42d4-ad15-3ececb2f0ff1 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:59.557117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958ad8f-194f-42d4-ad15-3ececb2f0ff1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:59.557427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958ad8f-194f-42d4-ad15-3ececb2f0ff1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:59.558859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958ad8f-194f-42d4-ad15-3ececb2f0ff1 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:59.565930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-594a5331-aede-40eb-a2c6-7ad4bf70b8b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:30:59.565930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:30:59.565930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:30:59.565930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:30:59.565930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:30:59.565930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:30:59.565930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:30:59.565930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:30:59.565930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:30:59.565930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:30:59.565930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:30:59.565930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:30:59.565930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:30:59.565930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:30:59.565930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:30:59.566956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:30:59.566956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:30:59.566956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:30:59.566956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:30:59.566956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:30:59.566956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:30:59.566956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:30:59.566956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:30:59.566956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:30:59.566956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:30:59.566956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:30:59.566956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:30:59.566956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:30:59.566956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:30:59.566956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:30:59.566956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:30:59.566956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:30:59.567889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:30:59.567889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:30:59.567889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:30:59.567889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:30:59.567889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 256/506] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:30:59.568146 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a5331-aede-40eb-a2c6-7ad4bf70b8b8 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:59.570350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:59.579947 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 251/507] 10.222.0.44 () {56 vars in 1137 bytes} [Thu Jul 21 05:30:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:59.580746 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958ad8f-194f-42d4-ad15-3ececb2f0ff1 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:59.670995 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a5331-aede-40eb-a2c6-7ad4bf70b8b8 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:59.679193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958ad8f-194f-42d4-ad15-3ececb2f0ff1 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:59.703111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0d855-7681-4e5c-ab6b-dd4c9291c156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:59.703778 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0d855-7681-4e5c-ab6b-dd4c9291c156 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:59.704056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0d855-7681-4e5c-ab6b-dd4c9291c156 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:59.704329 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0d855-7681-4e5c-ab6b-dd4c9291c156 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:59.705042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0d855-7681-4e5c-ab6b-dd4c9291c156 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:59.705607 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0d855-7681-4e5c-ab6b-dd4c9291c156 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:59.706719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0d855-7681-4e5c-ab6b-dd4c9291c156 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:59.713898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40181b29-7136-483b-9b48-ec68a20e30c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:59.714604 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40181b29-7136-483b-9b48-ec68a20e30c5 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:59.714890 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40181b29-7136-483b-9b48-ec68a20e30c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:59.715194 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40181b29-7136-483b-9b48-ec68a20e30c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:59.715951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40181b29-7136-483b-9b48-ec68a20e30c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:59.716256 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40181b29-7136-483b-9b48-ec68a20e30c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:59.717629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40181b29-7136-483b-9b48-ec68a20e30c5 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:59.719132 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 257/508] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:59.723668 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0d855-7681-4e5c-ab6b-dd4c9291c156 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:59.730418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 252/509] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:30:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:59.730418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40181b29-7136-483b-9b48-ec68a20e30c5 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:59.744062 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:59.787275 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:59.822388 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0d855-7681-4e5c-ab6b-dd4c9291c156 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:59.827883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40181b29-7136-483b-9b48-ec68a20e30c5 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:59.857310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23831c3e-6334-4de3-923e-650ce1d0807d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:59.857968 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23831c3e-6334-4de3-923e-650ce1d0807d None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:59.858253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23831c3e-6334-4de3-923e-650ce1d0807d None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:59.858530 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23831c3e-6334-4de3-923e-650ce1d0807d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:59.860253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2079e7-2637-4249-a065-b90846bb2f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:30:59.860889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2079e7-2637-4249-a065-b90846bb2f1e None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:30:59.861188 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2079e7-2637-4249-a065-b90846bb2f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:30:59.861459 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2079e7-2637-4249-a065-b90846bb2f1e None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:30:59.862179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2079e7-2637-4249-a065-b90846bb2f1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:30:59.862490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2079e7-2637-4249-a065-b90846bb2f1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:59.863909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2079e7-2637-4249-a065-b90846bb2f1e None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:59.876596 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 253/510] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:30:59.877392 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2079e7-2637-4249-a065-b90846bb2f1e None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:30:59.926196 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23831c3e-6334-4de3-923e-650ce1d0807d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:30:59.927754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23831c3e-6334-4de3-923e-650ce1d0807d None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:30:59.948945 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:30:59.972992 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2079e7-2637-4249-a065-b90846bb2f1e None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:30:59.992153 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:00.007363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc303ea5-e61e-4230-94c7-4eb92367ce8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:00.008020 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc303ea5-e61e-4230-94c7-4eb92367ce8b None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:00.008301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc303ea5-e61e-4230-94c7-4eb92367ce8b None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:00.008579 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc303ea5-e61e-4230-94c7-4eb92367ce8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:00.026166 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 258/511] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:31:00.027039 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23831c3e-6334-4de3-923e-650ce1d0807d None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:00.082351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc303ea5-e61e-4230-94c7-4eb92367ce8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:00.083767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc303ea5-e61e-4230-94c7-4eb92367ce8b None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:00.121833 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23831c3e-6334-4de3-923e-650ce1d0807d None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:00.141127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:00.154837 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4ab59-e5f9-423a-bf61-13dbeaa26008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:00.155664 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4ab59-e5f9-423a-bf61-13dbeaa26008 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:00.155943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4ab59-e5f9-423a-bf61-13dbeaa26008 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:00.156217 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4ab59-e5f9-423a-bf61-13dbeaa26008 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:00.163466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4ab59-e5f9-423a-bf61-13dbeaa26008 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:00.164982 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4ab59-e5f9-423a-bf61-13dbeaa26008 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:00.178884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-ecc4ab59-e5f9-423a-bf61-13dbeaa26008 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 05:31:00.178884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:00.178884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:00.178884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:00.178884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:00.178884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:00.178884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:00.178884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:00.178884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:00.178884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:00.178884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:00.178884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:00.178884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:00.178884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:00.178884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:00.178884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:00.179850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:00.180741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:00.180741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:00.180741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:00.180741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:00.180741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:00.180741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:00.180741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:00.180741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:00.180741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 05:31:00.180741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:00.180741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 259/512] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:31:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:00.181322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4ab59-e5f9-423a-bf61-13dbeaa26008 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:00.181395 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 254/513] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:30:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:31:00.182142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc303ea5-e61e-4230-94c7-4eb92367ce8b None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:00.203563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:00.282501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc303ea5-e61e-4230-94c7-4eb92367ce8b None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:00.283173 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4ab59-e5f9-423a-bf61-13dbeaa26008 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:00.315545 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fefaa-180a-4185-933a-f143e898f1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:00.316208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fefaa-180a-4185-933a-f143e898f1ce None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:00.316487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fefaa-180a-4185-933a-f143e898f1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:00.316758 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fefaa-180a-4185-933a-f143e898f1ce None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:00.317225 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff038ed-553a-4843-aeb0-a79244c9bd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:00.317457 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335fefaa-180a-4185-933a-f143e898f1ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:00.318027 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff038ed-553a-4843-aeb0-a79244c9bd41 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:00.318306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff038ed-553a-4843-aeb0-a79244c9bd41 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:00.318578 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff038ed-553a-4843-aeb0-a79244c9bd41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:00.318902 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335fefaa-180a-4185-933a-f143e898f1ce None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:00.328797 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff038ed-553a-4843-aeb0-a79244c9bd41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:00.330631 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff038ed-553a-4843-aeb0-a79244c9bd41 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:00.351122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 255/514] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:31:00 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:31:00.351122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fefaa-180a-4185-933a-f143e898f1ce None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:00.351486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-eff038ed-553a-4843-aeb0-a79244c9bd41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:00.351486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:00.351486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:00.351486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:00.351486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:00.351486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:00.351486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:00.351486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:00.351486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:00.351486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:00.351486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:00.351486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:00.351486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:00.351486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:00.351486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:00.352250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:00.352250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:00.352250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:00.352250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:00.352250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:00.352250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:00.352250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:00.352250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:00.352250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:00.352250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:00.352250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:00.352250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:00.352250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:00.352250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:00.352250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:00.352250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:00.352250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:00.353104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:00.353104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:00.353104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:00.353104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:00.353104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 260/515] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:00.353104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff038ed-553a-4843-aeb0-a79244c9bd41 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:00.387320 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:00.441560 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:00.468285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff038ed-553a-4843-aeb0-a79244c9bd41 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:00.474846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fefaa-180a-4185-933a-f143e898f1ce None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:00.507462 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894e56c-504b-4938-aabd-4542029dd672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:00.508187 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894e56c-504b-4938-aabd-4542029dd672 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:00.508472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894e56c-504b-4938-aabd-4542029dd672 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:00.508749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894e56c-504b-4938-aabd-4542029dd672 None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:00.509596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894e56c-504b-4938-aabd-4542029dd672 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:00.509905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894e56c-504b-4938-aabd-4542029dd672 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:00.511889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1ca38-1790-4ad0-b73d-99c1a0682f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:00.512092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894e56c-504b-4938-aabd-4542029dd672 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:00.512693 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1ca38-1790-4ad0-b73d-99c1a0682f0f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:00.512977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1ca38-1790-4ad0-b73d-99c1a0682f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:00.513247 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1ca38-1790-4ad0-b73d-99c1a0682f0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:00.525130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1ca38-1790-4ad0-b73d-99c1a0682f0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:00.526599 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1ca38-1790-4ad0-b73d-99c1a0682f0f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:00.546745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-5fa1ca38-1790-4ad0-b73d-99c1a0682f0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:00.546745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:00.546745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:00.546745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:00.546745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:00.546745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:00.546745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:00.546745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:00.546745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:00.546745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:00.546745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:00.546745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:00.546745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:00.546745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:00.546745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:00.547755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:00.547755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:00.547755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:00.547755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:00.547755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:00.547755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:00.547755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:00.547755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:00.547755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:00.547755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:00.547755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:00.547755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:00.547755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:00.547755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:00.547755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:00.547755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:00.547755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:00.548606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:00.548606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:00.548606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:00.548606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:00.548606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 256/516] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:00.549257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1ca38-1790-4ad0-b73d-99c1a0682f0f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:00.550670 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 261/517] 10.222.0.44 () {56 vars in 1137 bytes} [Thu Jul 21 05:31:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:00.551577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894e56c-504b-4938-aabd-4542029dd672 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:00.643553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:00.656575 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894e56c-504b-4938-aabd-4542029dd672 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:00.659696 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1ca38-1790-4ad0-b73d-99c1a0682f0f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:00.686438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:00.694676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4310d-f466-48f5-a5e2-a765bb94931e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:00.695420 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4310d-f466-48f5-a5e2-a765bb94931e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:00.695708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4310d-f466-48f5-a5e2-a765bb94931e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:00.695977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330645b-8963-4a82-9c7a-e4e43c708913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:00.696085 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4310d-f466-48f5-a5e2-a765bb94931e None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:00.696707 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4310d-f466-48f5-a5e2-a765bb94931e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:00.696799 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330645b-8963-4a82-9c7a-e4e43c708913 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:00.697019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4310d-f466-48f5-a5e2-a765bb94931e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:00.697118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330645b-8963-4a82-9c7a-e4e43c708913 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:00.697296 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330645b-8963-4a82-9c7a-e4e43c708913 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:00.698420 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c4310d-f466-48f5-a5e2-a765bb94931e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:00.708444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330645b-8963-4a82-9c7a-e4e43c708913 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:00.709894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330645b-8963-4a82-9c7a-e4e43c708913 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:00.710923 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 262/518] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:00.711621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4310d-f466-48f5-a5e2-a765bb94931e None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:00.731881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-b330645b-8963-4a82-9c7a-e4e43c708913 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:00.731881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:00.731881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:00.731881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:00.731881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:00.731881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:00.731881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:00.731881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:00.731881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:00.731881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:00.731881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:00.731881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:00.731881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:00.731881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:00.731881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:00.732802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:00.732802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:00.732802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:00.732802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:00.732802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:00.732802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:00.732802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:00.732802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:00.732802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:00.732802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:00.732802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:00.732802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:00.732802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:00.732802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:00.732802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:00.732802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:00.732802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:00.733661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:00.733661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:00.733661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:00.733661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:00.737708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 257/519] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:00.738563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330645b-8963-4a82-9c7a-e4e43c708913 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:00.815554 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4310d-f466-48f5-a5e2-a765bb94931e None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:00.842543 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330645b-8963-4a82-9c7a-e4e43c708913 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:00.851564 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1604c3-3a51-41c2-9908-92bf9b94c2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:00.852338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1604c3-3a51-41c2-9908-92bf9b94c2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:00.852622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1604c3-3a51-41c2-9908-92bf9b94c2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:00.852888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1604c3-3a51-41c2-9908-92bf9b94c2b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:00.853713 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1604c3-3a51-41c2-9908-92bf9b94c2b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:00.854022 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1604c3-3a51-41c2-9908-92bf9b94c2b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:00.855430 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1604c3-3a51-41c2-9908-92bf9b94c2b8 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:00.875098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 263/520] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:00.875098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1604c3-3a51-41c2-9908-92bf9b94c2b8 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:00.878685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93123573-d80e-4755-ac9e-e02d698993d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:00.879365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93123573-d80e-4755-ac9e-e02d698993d0 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:00.879646 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93123573-d80e-4755-ac9e-e02d698993d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:00.879915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93123573-d80e-4755-ac9e-e02d698993d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:00.885462 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:00.933107 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:00.955906 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93123573-d80e-4755-ac9e-e02d698993d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:00.957387 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93123573-d80e-4755-ac9e-e02d698993d0 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:00.981274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1604c3-3a51-41c2-9908-92bf9b94c2b8 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:01.017296 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535f12c-ca79-4831-b066-abce240df6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:01.018214 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535f12c-ca79-4831-b066-abce240df6ec None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:01.018611 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535f12c-ca79-4831-b066-abce240df6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:01.018886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535f12c-ca79-4831-b066-abce240df6ec None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:01.026350 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535f12c-ca79-4831-b066-abce240df6ec None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:01.027983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535f12c-ca79-4831-b066-abce240df6ec None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:01.043632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-e535f12c-ca79-4831-b066-abce240df6ec None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 05:31:01.043632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:01.043632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:01.043632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:01.043632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:01.043632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:01.043632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:01.043632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:01.043632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:01.043632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:01.043632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:01.043632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:01.043632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:01.043632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:01.043632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:01.043632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:01.044547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:01.050576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:01.050576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:01.050576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:01.050576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:01.050576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:01.050576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:01.050576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:01.050576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:01.050576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 05:31:01.050576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:01.050576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 264/521] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:31:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:01.050576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535f12c-ca79-4831-b066-abce240df6ec None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:01.066439 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 258/522] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:31:01.067207 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93123573-d80e-4755-ac9e-e02d698993d0 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:01.128071 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:01.157030 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:01.161761 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535f12c-ca79-4831-b066-abce240df6ec None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:01.179193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93123573-d80e-4755-ac9e-e02d698993d0 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:01.200216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b90db9-34dd-4497-b373-3a40f6e67dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:01.200827 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b90db9-34dd-4497-b373-3a40f6e67dbc None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:01.201112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b90db9-34dd-4497-b373-3a40f6e67dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:01.201388 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b90db9-34dd-4497-b373-3a40f6e67dbc None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:01.202113 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b90db9-34dd-4497-b373-3a40f6e67dbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:01.202422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b90db9-34dd-4497-b373-3a40f6e67dbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:01.203851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b90db9-34dd-4497-b373-3a40f6e67dbc None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:01.214976 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc6caa-ca70-4938-b02e-c9f58178e495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:01.220140 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 265/523] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:01.220269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc6caa-ca70-4938-b02e-c9f58178e495 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:01.220269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc6caa-ca70-4938-b02e-c9f58178e495 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:01.220269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc6caa-ca70-4938-b02e-c9f58178e495 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:01.220495 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b90db9-34dd-4497-b373-3a40f6e67dbc None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:01.228516 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bc6caa-ca70-4938-b02e-c9f58178e495 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:01.230500 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bc6caa-ca70-4938-b02e-c9f58178e495 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:01.249948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-a5bc6caa-ca70-4938-b02e-c9f58178e495 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:01.249948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:01.249948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:01.249948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:01.249948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:01.249948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:01.249948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:01.249948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:01.249948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:01.249948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:01.249948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:01.249948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:01.249948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:01.249948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:01.249948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:01.250802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:01.250802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:01.250802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:01.250802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:01.250802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:01.250802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:01.250802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:01.250802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:01.250802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:01.250802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:01.250802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:01.250802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:01.250802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:01.250802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:01.250802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:01.250802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:01.250802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:01.251729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:01.251729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:01.251729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:01.251729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:01.251729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:01.251729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:01.251729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:01.251729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:01.256984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 259/524] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:01.256984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc6caa-ca70-4938-b02e-c9f58178e495 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:01.323610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b90db9-34dd-4497-b373-3a40f6e67dbc None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:01.352954 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:01.360352 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c97b6a-d732-4063-a020-a9c3ff735a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:01.361029 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c97b6a-d732-4063-a020-a9c3ff735a57 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:01.361317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c97b6a-d732-4063-a020-a9c3ff735a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:01.361578 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c97b6a-d732-4063-a020-a9c3ff735a57 None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:01.362411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c97b6a-d732-4063-a020-a9c3ff735a57 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:01.362718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c97b6a-d732-4063-a020-a9c3ff735a57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:01.364206 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c97b6a-d732-4063-a020-a9c3ff735a57 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:01.368107 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc6caa-ca70-4938-b02e-c9f58178e495 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:01.385968 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 266/525] 10.222.0.44 () {56 vars in 1139 bytes} [Thu Jul 21 05:31:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:01.385968 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c97b6a-d732-4063-a020-a9c3ff735a57 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:01.409820 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948c8bf-6254-485b-9f7a-d6ab48acdfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:01.410732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948c8bf-6254-485b-9f7a-d6ab48acdfd3 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:01.411033 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948c8bf-6254-485b-9f7a-d6ab48acdfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:01.411318 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948c8bf-6254-485b-9f7a-d6ab48acdfd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:01.421027 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:01.423673 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948c8bf-6254-485b-9f7a-d6ab48acdfd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:01.425805 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948c8bf-6254-485b-9f7a-d6ab48acdfd3 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:01.451114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-2948c8bf-6254-485b-9f7a-d6ab48acdfd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:01.451114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:01.451114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:01.451114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:01.451114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:01.451114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:01.451114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:01.451114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:01.451114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:01.451114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:01.451114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:01.451114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:01.451114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:01.451114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:01.451114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:01.452202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:01.452202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:01.452202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:01.452202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:01.452202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:01.452202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:01.452202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:01.452202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:01.452202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:01.452202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:01.452202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:01.452202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:01.452202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:01.452202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:01.452202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:01.452202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:01.452202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:01.453205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:01.453205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:01.453205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:01.453205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:01.453205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 260/526] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:01.453205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948c8bf-6254-485b-9f7a-d6ab48acdfd3 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:01.494788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c97b6a-d732-4063-a020-a9c3ff735a57 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:01.529120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da6bae-4a88-4757-befd-178b3efbd2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:01.530301 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da6bae-4a88-4757-befd-178b3efbd2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:01.530593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da6bae-4a88-4757-befd-178b3efbd2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:01.530867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da6bae-4a88-4757-befd-178b3efbd2d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:01.538906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da6bae-4a88-4757-befd-178b3efbd2d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:01.540412 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da6bae-4a88-4757-befd-178b3efbd2d9 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:01.554932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-37da6bae-4a88-4757-befd-178b3efbd2d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:01.554932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:01.554932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:01.554932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:01.554932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:01.554932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:01.554932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:01.554932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:01.554932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:01.554932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:01.554932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:01.554932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:01.554932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:01.554932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:01.554932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:01.556110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:01.556110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:01.556110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:01.556110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:01.556110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:01.556110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:01.556110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:01.556110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:01.556110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:01.556110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:01.556110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:01.556110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:01.556110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:01.556110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:01.556110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:01.556110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:01.556110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:01.556959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:01.556959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:01.556959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:01.556959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:01.556959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:01.556959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:01.556959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:01.556959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:01.556959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 267/527] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:01.556959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da6bae-4a88-4757-befd-178b3efbd2d9 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:01.558267 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948c8bf-6254-485b-9f7a-d6ab48acdfd3 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:01.601105 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d82ea6-05c6-4ca3-9c4e-18a6c1637450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:01.602126 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d82ea6-05c6-4ca3-9c4e-18a6c1637450 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:01.602553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d82ea6-05c6-4ca3-9c4e-18a6c1637450 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:01.602847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d82ea6-05c6-4ca3-9c4e-18a6c1637450 None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:01.603675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d82ea6-05c6-4ca3-9c4e-18a6c1637450 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:01.603980 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d82ea6-05c6-4ca3-9c4e-18a6c1637450 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:01.605440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d82ea6-05c6-4ca3-9c4e-18a6c1637450 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:01.607920 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:01.623764 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 261/528] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:01.624953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d82ea6-05c6-4ca3-9c4e-18a6c1637450 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:01.669596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da6bae-4a88-4757-befd-178b3efbd2d9 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:01.686763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:01.703634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cb824-f5e1-4b3a-96c0-1f55ac41a23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:01.704222 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cb824-f5e1-4b3a-96c0-1f55ac41a23d None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:01.704542 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cb824-f5e1-4b3a-96c0-1f55ac41a23d None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:01.704818 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cb824-f5e1-4b3a-96c0-1f55ac41a23d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:01.736491 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d82ea6-05c6-4ca3-9c4e-18a6c1637450 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:01.772112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018b593-254e-4bb7-931e-a3588ef23994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:01.772277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973cb824-f5e1-4b3a-96c0-1f55ac41a23d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:01.772781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018b593-254e-4bb7-931e-a3588ef23994 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:01.773139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018b593-254e-4bb7-931e-a3588ef23994 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:01.773419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018b593-254e-4bb7-931e-a3588ef23994 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:01.773650 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973cb824-f5e1-4b3a-96c0-1f55ac41a23d None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:01.774156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018b593-254e-4bb7-931e-a3588ef23994 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:01.774464 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018b593-254e-4bb7-931e-a3588ef23994 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:01.776750 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018b593-254e-4bb7-931e-a3588ef23994 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:01.791419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 262/529] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:01.792573 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018b593-254e-4bb7-931e-a3588ef23994 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:01.826136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:01.878184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 268/530] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:31:01.883350 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cb824-f5e1-4b3a-96c0-1f55ac41a23d None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:01.898447 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018b593-254e-4bb7-931e-a3588ef23994 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:01.912230 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:01.935382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b3d04-71f1-4c06-b555-c92e08e19719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:01.936049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b3d04-71f1-4c06-b555-c92e08e19719 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:01.936329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b3d04-71f1-4c06-b555-c92e08e19719 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:01.936607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b3d04-71f1-4c06-b555-c92e08e19719 None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:01.937399 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b3d04-71f1-4c06-b555-c92e08e19719 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:01.937708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b3d04-71f1-4c06-b555-c92e08e19719 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:01.939195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b3d04-71f1-4c06-b555-c92e08e19719 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:01.951342 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 263/531] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:01.951342 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b3d04-71f1-4c06-b555-c92e08e19719 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:01.992600 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cb824-f5e1-4b3a-96c0-1f55ac41a23d None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:02.027561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fed8a7-e8e3-43ee-96da-29739b2def03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:02.028225 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fed8a7-e8e3-43ee-96da-29739b2def03 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:02.028506 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fed8a7-e8e3-43ee-96da-29739b2def03 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:02.028780 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fed8a7-e8e3-43ee-96da-29739b2def03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:02.029610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fed8a7-e8e3-43ee-96da-29739b2def03 None admin] RBAC: Adding query filter params (role.id=bb94781ab8054924a5cb21b561b12bbb, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=928d014648cd41d7b58e7e01845eb748) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:02.029924 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fed8a7-e8e3-43ee-96da-29739b2def03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:02.031415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fed8a7-e8e3-43ee-96da-29739b2def03 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:02.038084 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 269/532] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:01 2022] GET /identity/v3/role_assignments?user.id=928d014648cd41d7b58e7e01845eb748&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=bb94781ab8054924a5cb21b561b12bbb => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:02.038811 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fed8a7-e8e3-43ee-96da-29739b2def03 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:02.056715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:02.069775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b3d04-71f1-4c06-b555-c92e08e19719 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:02.110238 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618b151-d9a5-448f-9def-a42ef554c568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:02.111171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618b151-d9a5-448f-9def-a42ef554c568 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:02.111456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618b151-d9a5-448f-9def-a42ef554c568 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:02.111789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618b151-d9a5-448f-9def-a42ef554c568 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:02.119729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618b151-d9a5-448f-9def-a42ef554c568 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:02.121237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4618b151-d9a5-448f-9def-a42ef554c568 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:02.136915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-4618b151-d9a5-448f-9def-a42ef554c568 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:02.136915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:02.136915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:02.136915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:02.136915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:02.136915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:02.136915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:02.136915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:02.136915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:02.136915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:02.136915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:02.136915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:02.136915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:02.136915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:02.136915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:02.137852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:02.137852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:02.137852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:02.137852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:02.137852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:02.137852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:02.137852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:02.137852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:02.137852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:02.137852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:02.137852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:02.137852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:02.137852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:02.137852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:02.137852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:02.137852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:02.137852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:02.138695 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:02.138695 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:02.138695 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:02.138695 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:02.138695 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:02.138695 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:02.138695 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:02.138695 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:02.143317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 264/533] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:02.144366 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618b151-d9a5-448f-9def-a42ef554c568 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:02.169712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fed8a7-e8e3-43ee-96da-29739b2def03 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:02.181681 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:02.210355 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfe164-bdfd-4a7a-a83e-f957951d5f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:02.211646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfe164-bdfd-4a7a-a83e-f957951d5f8a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:02.211917 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfe164-bdfd-4a7a-a83e-f957951d5f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:02.212194 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfe164-bdfd-4a7a-a83e-f957951d5f8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:02.223198 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfe164-bdfd-4a7a-a83e-f957951d5f8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:02.224697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfe164-bdfd-4a7a-a83e-f957951d5f8a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:02.242881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-16dfe164-bdfd-4a7a-a83e-f957951d5f8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:02.242881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:02.242881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:02.242881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:02.242881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:02.242881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:02.242881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:02.242881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:02.242881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:02.242881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:02.242881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:02.242881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:02.242881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:02.242881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:02.242881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:02.243783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:02.243783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:02.243783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:02.243783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:02.243783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:02.243783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:02.243783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:02.243783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:02.243783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:02.243783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:02.243783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:02.243783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:02.243783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:02.243783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:02.243783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:02.243783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:02.243783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:02.245073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:02.245073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:02.245073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:02.245073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:02.245073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 270/534] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:02.245341 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfe164-bdfd-4a7a-a83e-f957951d5f8a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:02.265690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618b151-d9a5-448f-9def-a42ef554c568 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:02.310546 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:02.313977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1bec5-99e3-4b83-99eb-c70b5cc3afeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:02.314687 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1bec5-99e3-4b83-99eb-c70b5cc3afeb None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:02.315152 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1bec5-99e3-4b83-99eb-c70b5cc3afeb None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:02.315589 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1bec5-99e3-4b83-99eb-c70b5cc3afeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:02.317829 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1bec5-99e3-4b83-99eb-c70b5cc3afeb None admin] RBAC: Adding query filter params (role.id=bb94781ab8054924a5cb21b561b12bbb, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=2707c6967a0a419a98ae142ff1d1a961) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:02.318333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1bec5-99e3-4b83-99eb-c70b5cc3afeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:02.320264 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1bec5-99e3-4b83-99eb-c70b5cc3afeb None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:02.328466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 265/535] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:02 2022] GET /identity/v3/role_assignments?user.id=2707c6967a0a419a98ae142ff1d1a961&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=bb94781ab8054924a5cb21b561b12bbb => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:02.329347 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1bec5-99e3-4b83-99eb-c70b5cc3afeb None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:02.393079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfe164-bdfd-4a7a-a83e-f957951d5f8a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:02.440776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8195d01-29c2-4648-977f-d25bd4b73680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:02.441469 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8195d01-29c2-4648-977f-d25bd4b73680 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:02.441750 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8195d01-29c2-4648-977f-d25bd4b73680 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:02.442040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8195d01-29c2-4648-977f-d25bd4b73680 None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:02.442852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8195d01-29c2-4648-977f-d25bd4b73680 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:02.443199 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8195d01-29c2-4648-977f-d25bd4b73680 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:02.444983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8195d01-29c2-4648-977f-d25bd4b73680 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:02.448965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:02.456944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1bec5-99e3-4b83-99eb-c70b5cc3afeb None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:02.468124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 271/536] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:02.469583 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600bba6-ea4c-413d-b305-3c7936b09caf None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:02.469879 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600bba6-ea4c-413d-b305-3c7936b09caf None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:02.470158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600bba6-ea4c-413d-b305-3c7936b09caf None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:02.475877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 272/537] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:02.477040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600bba6-ea4c-413d-b305-3c7936b09caf None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:02.509917 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28382107-c3d2-4de4-96a1-95f1b37c3139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:02.510894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28382107-c3d2-4de4-96a1-95f1b37c3139 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:02.511275 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28382107-c3d2-4de4-96a1-95f1b37c3139 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:02.511623 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28382107-c3d2-4de4-96a1-95f1b37c3139 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:02.513113 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28382107-c3d2-4de4-96a1-95f1b37c3139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:02.513751 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28382107-c3d2-4de4-96a1-95f1b37c3139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:02.515451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28382107-c3d2-4de4-96a1-95f1b37c3139 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:02.530256 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 266/538] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:02.531707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f64c7b-0fb8-4f85-b986-99246da2ce81 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:02.532068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f64c7b-0fb8-4f85-b986-99246da2ce81 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:02.532522 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f64c7b-0fb8-4f85-b986-99246da2ce81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:02.549591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-37f64c7b-0fb8-4f85-b986-99246da2ce81 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:02.550778 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-37f64c7b-0fb8-4f85-b986-99246da2ce81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:02.596720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600bba6-ea4c-413d-b305-3c7936b09caf None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:02.627491 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:02.639109 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 267/539] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:31:02.640019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f64c7b-0fb8-4f85-b986-99246da2ce81 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:02.642915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe533fe-bdc7-4b2f-aae1-a5715402659d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:02.643690 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe533fe-bdc7-4b2f-aae1-a5715402659d None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:02.643978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe533fe-bdc7-4b2f-aae1-a5715402659d None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:02.644284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe533fe-bdc7-4b2f-aae1-a5715402659d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:02.645123 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe533fe-bdc7-4b2f-aae1-a5715402659d None admin] RBAC: Adding query filter params (role.id=bb94781ab8054924a5cb21b561b12bbb, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=f3f76371c6a04fc5a477d6ef1908a08e) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:02.645429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe533fe-bdc7-4b2f-aae1-a5715402659d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:02.648763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe533fe-bdc7-4b2f-aae1-a5715402659d None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:02.667228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 273/540] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:02 2022] GET /identity/v3/role_assignments?user.id=f3f76371c6a04fc5a477d6ef1908a08e&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=bb94781ab8054924a5cb21b561b12bbb => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:02.671179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe533fe-bdc7-4b2f-aae1-a5715402659d None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:02.828975 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:02.863841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f64c7b-0fb8-4f85-b986-99246da2ce81 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:02.873189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe533fe-bdc7-4b2f-aae1-a5715402659d None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:02.925586 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc238a01-d315-4947-b71e-762382853430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:02.927483 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc238a01-d315-4947-b71e-762382853430 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:02.928163 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc238a01-d315-4947-b71e-762382853430 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:02.928814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04591253-8d33-4689-87d6-d477da689833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:02.929103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc238a01-d315-4947-b71e-762382853430 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:02.930266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04591253-8d33-4689-87d6-d477da689833 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:02.930826 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04591253-8d33-4689-87d6-d477da689833 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:02.931389 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04591253-8d33-4689-87d6-d477da689833 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:02.949007 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc238a01-d315-4947-b71e-762382853430 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:02.952147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04591253-8d33-4689-87d6-d477da689833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:02.959356 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc238a01-d315-4947-b71e-762382853430 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:02.977411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-04591253-8d33-4689-87d6-d477da689833 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:02.977411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:02.977411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:02.977411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:02.977411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:02.977411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:02.977411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:02.977411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:02.977411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:02.977411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:02.977411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:02.977411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:02.977411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:02.977411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:02.977411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:02.977411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:02.979104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:02.979104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:02.979104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:02.979104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:02.979104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:02.979104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:02.979104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:02.979104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:02.979104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:02.979104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:02.979104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:02.979104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:02.979104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:02.979104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:02.979104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:02.979104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:02.980383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:02.980383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:02.980383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:02.980383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:02.980383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:02.980383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:02.980383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:02.980383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:02.980383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:02.980383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:02.980383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:02.980383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:02.980383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 274/541] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:31:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:02.988055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-fc238a01-d315-4947-b71e-762382853430 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:02.988055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:02.988055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:02.988055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:02.988055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:02.988055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:02.988055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:02.988055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:02.988055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:02.988055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:02.988055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:02.988055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:02.988055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:02.988055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:02.988055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:02.989965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:02.989965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:02.989965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:02.989965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:02.989965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:02.989965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:02.989965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:02.989965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:02.989965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:02.989965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:02.989965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:02.989965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:02.989965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:02.989965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:02.989965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:02.989965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:02.989965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:02.992118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:02.992118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:02.992118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:02.992118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:02.992118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:02.992118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:02.992118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:02.992118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:02.993307 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04591253-8d33-4689-87d6-d477da689833 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:02.995614 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 268/542] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:02.996619 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc238a01-d315-4947-b71e-762382853430 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:03.077316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:03.134841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc238a01-d315-4947-b71e-762382853430 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:03.170109 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca888276-af6e-4047-88d3-a64442c1299f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:03.170936 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca888276-af6e-4047-88d3-a64442c1299f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:03.171483 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca888276-af6e-4047-88d3-a64442c1299f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:03.171863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca888276-af6e-4047-88d3-a64442c1299f None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:03.172789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca888276-af6e-4047-88d3-a64442c1299f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:03.173212 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca888276-af6e-4047-88d3-a64442c1299f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:03.174755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca888276-af6e-4047-88d3-a64442c1299f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:03.187715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04591253-8d33-4689-87d6-d477da689833 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:03.188288 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 269/543] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:03.194339 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca888276-af6e-4047-88d3-a64442c1299f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:03.205915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:03.247182 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747b5be-bca5-40b8-886a-70fc52736545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:03.247863 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747b5be-bca5-40b8-886a-70fc52736545 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:03.248141 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747b5be-bca5-40b8-886a-70fc52736545 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:03.248411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747b5be-bca5-40b8-886a-70fc52736545 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:03.249165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747b5be-bca5-40b8-886a-70fc52736545 None admin] RBAC: Adding query filter params (name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:03.249466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747b5be-bca5-40b8-886a-70fc52736545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:03.250846 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747b5be-bca5-40b8-886a-70fc52736545 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:03.266534 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 275/544] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:31:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:03.271398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747b5be-bca5-40b8-886a-70fc52736545 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:03.376080 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747b5be-bca5-40b8-886a-70fc52736545 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:03.396991 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca888276-af6e-4047-88d3-a64442c1299f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:03.408909 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:03.419430 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfa19c-c0cb-412b-b8f3-6fca6fa67015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:03.419430 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfa19c-c0cb-412b-b8f3-6fca6fa67015 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:03.419430 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfa19c-c0cb-412b-b8f3-6fca6fa67015 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:03.419430 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdfa19c-c0cb-412b-b8f3-6fca6fa67015 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:03.428823 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdfa19c-c0cb-412b-b8f3-6fca6fa67015 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:03.431949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdfa19c-c0cb-412b-b8f3-6fca6fa67015 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:03.450627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-dcdfa19c-c0cb-412b-b8f3-6fca6fa67015 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:03.450627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:03.450627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:03.450627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:03.450627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:03.450627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:03.450627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:03.450627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:03.450627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:03.450627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:03.450627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:03.450627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:03.450627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:03.450627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:03.450627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:03.451524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:03.451524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:03.451524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:03.451524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:03.451524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:03.451524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:03.451524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:03.451524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:03.451524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:03.451524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:03.451524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:03.451524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:03.451524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:03.451524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:03.451524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:03.451524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:03.451524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:03.452352 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:03.452352 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:03.452352 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:03.452352 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:03.456357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 276/545] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:03.458154 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfa19c-c0cb-412b-b8f3-6fca6fa67015 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:03.492269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7460ee-f727-41af-931b-4c888f99998b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:03.493185 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7460ee-f727-41af-931b-4c888f99998b None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:03.493607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7460ee-f727-41af-931b-4c888f99998b None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:03.494004 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7460ee-f727-41af-931b-4c888f99998b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:03.494974 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7460ee-f727-41af-931b-4c888f99998b None admin] RBAC: Adding query filter params (role.id=bb94781ab8054924a5cb21b561b12bbb, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=e07508a9fa9146528620b7f8abec8549) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:03.495445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7460ee-f727-41af-931b-4c888f99998b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:03.497053 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7460ee-f727-41af-931b-4c888f99998b None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:03.511102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 270/546] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:03 2022] GET /identity/v3/role_assignments?user.id=e07508a9fa9146528620b7f8abec8549&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=bb94781ab8054924a5cb21b561b12bbb => generated 259 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:03.563259 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdfa19c-c0cb-412b-b8f3-6fca6fa67015 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:03.601946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331e5b5-7982-438b-a2d6-5c41510ec30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:03.602645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331e5b5-7982-438b-a2d6-5c41510ec30e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:03.602925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331e5b5-7982-438b-a2d6-5c41510ec30e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:03.603233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331e5b5-7982-438b-a2d6-5c41510ec30e None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:03.603978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331e5b5-7982-438b-a2d6-5c41510ec30e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:03.604285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331e5b5-7982-438b-a2d6-5c41510ec30e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:03.605727 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331e5b5-7982-438b-a2d6-5c41510ec30e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:03.624624 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 277/547] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:03.626063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7460ee-f727-41af-931b-4c888f99998b None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:03.729125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:03.764085 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7460ee-f727-41af-931b-4c888f99998b None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:03.799004 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f1793-0931-411c-8fd2-6317fd866fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:03.799840 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f1793-0931-411c-8fd2-6317fd866fab None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:03.800129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f1793-0931-411c-8fd2-6317fd866fab None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:03.800410 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f1793-0931-411c-8fd2-6317fd866fab None admin] PATH_INFO: `/v3/users/swift` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:03.808261 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f1793-0931-411c-8fd2-6317fd866fab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:03.809727 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f1793-0931-411c-8fd2-6317fd866fab None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:03.825001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-197f1793-0931-411c-8fd2-6317fd866fab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 05:31:03.825001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:03.825001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:03.825001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:03.825001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:03.825001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:03.825001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:03.825001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:03.825001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:03.825001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:03.825001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:03.825001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:03.825001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:03.825001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:03.825001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:03.825001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:03.825882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:03.827321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:03.827321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:03.827321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:03.827321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:03.827321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:03.827321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:03.827321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:03.827321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:03.827321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 05:31:03.827321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:03.831529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 271/548] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:31:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:03.837748 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331e5b5-7982-438b-a2d6-5c41510ec30e None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:03.874771 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:03.989785 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331e5b5-7982-438b-a2d6-5c41510ec30e None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:04.043787 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ed156-d57f-42a0-b91c-104e5a48751b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:04.044473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ed156-d57f-42a0-b91c-104e5a48751b None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:04.044761 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ed156-d57f-42a0-b91c-104e5a48751b None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:04.045037 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ed156-d57f-42a0-b91c-104e5a48751b None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:04.045876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ed156-d57f-42a0-b91c-104e5a48751b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:04.046183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ed156-d57f-42a0-b91c-104e5a48751b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:04.047683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ed156-d57f-42a0-b91c-104e5a48751b None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:04.075771 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 278/549] 10.222.0.44 () {56 vars in 1135 bytes} [Thu Jul 21 05:31:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:04.081339 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f1793-0931-411c-8fd2-6317fd866fab None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:04.201099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f1793-0931-411c-8fd2-6317fd866fab None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:04.239042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:04.246699 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d09b5-c438-4211-83dd-8e9378c04d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:04.247903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d09b5-c438-4211-83dd-8e9378c04d12 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:04.248395 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d09b5-c438-4211-83dd-8e9378c04d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:04.248861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d09b5-c438-4211-83dd-8e9378c04d12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:04.326665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d09b5-c438-4211-83dd-8e9378c04d12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:04.328470 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d09b5-c438-4211-83dd-8e9378c04d12 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:04.434658 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 272/550] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:31:04.440132 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ed156-d57f-42a0-b91c-104e5a48751b None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:04.547052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:04.568897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ed156-d57f-42a0-b91c-104e5a48751b None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:04.593073 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438b1fc-c070-4b40-9ef4-884fb8a8b53c None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:04.593379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438b1fc-c070-4b40-9ef4-884fb8a8b53c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:04.593632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438b1fc-c070-4b40-9ef4-884fb8a8b53c None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:04.595200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 273/551] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:31:04.601000 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d99b9-51f0-41a1-b89b-bfe745d3fb37 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:04.601489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d99b9-51f0-41a1-b89b-bfe745d3fb37 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:04.601669 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d99b9-51f0-41a1-b89b-bfe745d3fb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:04.627244 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0bcd0-a4a2-48a2-a1e3-c3d3cb6baa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:04.628078 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0bcd0-a4a2-48a2-a1e3-c3d3cb6baa1e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:04.628361 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0bcd0-a4a2-48a2-a1e3-c3d3cb6baa1e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:04.628634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0bcd0-a4a2-48a2-a1e3-c3d3cb6baa1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:04.630132 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-d41d99b9-51f0-41a1-b89b-bfe745d3fb37 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:04.632940 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d41d99b9-51f0-41a1-b89b-bfe745d3fb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:04.655779 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb0bcd0-a4a2-48a2-a1e3-c3d3cb6baa1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:04.659692 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb0bcd0-a4a2-48a2-a1e3-c3d3cb6baa1e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:04.683404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-2cb0bcd0-a4a2-48a2-a1e3-c3d3cb6baa1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:04.683404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:04.683404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:04.683404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:04.683404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:04.683404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:04.683404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:04.683404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:04.683404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:04.683404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:04.683404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:04.683404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:04.683404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:04.683404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:04.683404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:04.684618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:04.684618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:04.684618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:04.684618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:04.684618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:04.684618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:04.684618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:04.684618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:04.684618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:04.684618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:04.684618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:04.684618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:04.684618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:04.684618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:04.684618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:04.684618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:04.684618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:04.685640 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:04.685640 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:04.685640 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:04.685640 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:04.686981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 279/552] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:04.694319 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:04.698544 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0bcd0-a4a2-48a2-a1e3-c3d3cb6baa1e None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:04.812549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0bcd0-a4a2-48a2-a1e3-c3d3cb6baa1e None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:04.827044 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 274/553] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:31:04.854147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:04.861556 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f972509-2524-45bd-80f2-be9981880215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:04.862513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f972509-2524-45bd-80f2-be9981880215 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:04.862993 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f972509-2524-45bd-80f2-be9981880215 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:04.863493 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f972509-2524-45bd-80f2-be9981880215 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:04.864435 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f972509-2524-45bd-80f2-be9981880215 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:04.864930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f972509-2524-45bd-80f2-be9981880215 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:04.866521 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f972509-2524-45bd-80f2-be9981880215 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:04.886385 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 280/554] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:04.889715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f972509-2524-45bd-80f2-be9981880215 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:04.889847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d99b9-51f0-41a1-b89b-bfe745d3fb37 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:05.014899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d99b9-51f0-41a1-b89b-bfe745d3fb37 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:05.021465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f972509-2524-45bd-80f2-be9981880215 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:05.050991 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64088c-1ee9-440b-8750-a4d87e3fc512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:05.051921 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64088c-1ee9-440b-8750-a4d87e3fc512 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:05.052205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64088c-1ee9-440b-8750-a4d87e3fc512 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:05.052481 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64088c-1ee9-440b-8750-a4d87e3fc512 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:05.060163 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c64088c-1ee9-440b-8750-a4d87e3fc512 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:05.061638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c64088c-1ee9-440b-8750-a4d87e3fc512 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:05.064817 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db4af1-7c47-408d-ad89-d4f6123206a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:05.065943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db4af1-7c47-408d-ad89-d4f6123206a6 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:05.066427 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db4af1-7c47-408d-ad89-d4f6123206a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:05.066894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db4af1-7c47-408d-ad89-d4f6123206a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:05.076084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-3c64088c-1ee9-440b-8750-a4d87e3fc512 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:05.076084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:05.076084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:05.076084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:05.076084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:05.076084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:05.076084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:05.076084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:05.076084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:05.076084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:05.076084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:05.076084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:05.076084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:05.076084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:05.076084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:05.076972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:05.076972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:05.076972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:05.076972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:05.076972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:05.076972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:05.076972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:05.076972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:05.076972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:05.076972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:05.076972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:05.076972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:05.076972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:05.076972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:05.076972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:05.076972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:05.076972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:05.077842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:05.077842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:05.077842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:05.077842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:05.077842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:05.077842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:05.077842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:05.077842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:05.080455 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31db4af1-7c47-408d-ad89-d4f6123206a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:05.082149 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 275/555] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:05.083550 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92ae6d-24ad-4323-bf6c-e092551afb0c None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:05.083840 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92ae6d-24ad-4323-bf6c-e092551afb0c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:05.084119 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92ae6d-24ad-4323-bf6c-e092551afb0c None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:05.085728 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 276/556] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:05.086428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92ae6d-24ad-4323-bf6c-e092551afb0c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:05.090492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-31db4af1-7c47-408d-ad89-d4f6123206a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:05.090492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:05.090492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:05.090492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:05.090492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:05.090492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:05.090492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:05.090492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:05.090492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:05.090492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:05.090492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:05.090492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:05.090492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:05.090492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:05.090492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:05.090492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:05.091466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:05.091466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:05.091466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:05.091466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:05.091466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:05.091466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:05.091466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:05.091466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:05.091466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:05.091466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:05.091466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:05.091466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:05.091466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:05.091466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:05.091466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:05.091466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:05.092831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:05.092831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:05.092831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:05.092831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:05.092831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:05.092831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:05.092831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:05.092831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:05.092831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:05.092831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:05.092831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:05.092831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:05.092831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 281/557] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:31:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:05.093533 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79c6b0-851f-46cb-8312-3c9d8df46ee2 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:05.093610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79c6b0-851f-46cb-8312-3c9d8df46ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:05.094241 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79c6b0-851f-46cb-8312-3c9d8df46ee2 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:05.095417 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 282/558] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:05.096447 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fbf31-4445-4635-848d-9335d57f5368 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:05.096728 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fbf31-4445-4635-848d-9335d57f5368 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:05.097311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fbf31-4445-4635-848d-9335d57f5368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:05.116078 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-ec7fbf31-4445-4635-848d-9335d57f5368 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:05.117413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ec7fbf31-4445-4635-848d-9335d57f5368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:05.119515 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:05.166982 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:05.195082 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 283/559] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:31:05.195082 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7fbf31-4445-4635-848d-9335d57f5368 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:05.207963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92ae6d-24ad-4323-bf6c-e092551afb0c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:05.244762 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f69e53-302e-4957-836b-589faa618524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:05.245664 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f69e53-302e-4957-836b-589faa618524 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:05.246086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f69e53-302e-4957-836b-589faa618524 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:05.246529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f69e53-302e-4957-836b-589faa618524 None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:05.247651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f69e53-302e-4957-836b-589faa618524 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:05.248100 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f69e53-302e-4957-836b-589faa618524 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:05.249831 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f69e53-302e-4957-836b-589faa618524 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:05.263243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 277/560] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:05.264831 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ef22d-8da9-4e8a-8049-79f71b5d1144 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:05.265111 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ef22d-8da9-4e8a-8049-79f71b5d1144 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:05.265382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ef22d-8da9-4e8a-8049-79f71b5d1144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:05.282554 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-324ef22d-8da9-4e8a-8049-79f71b5d1144 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:05.283793 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-324ef22d-8da9-4e8a-8049-79f71b5d1144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:05.299589 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7fbf31-4445-4635-848d-9335d57f5368 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:05.334465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7794e8-6d8f-4391-8666-41976104ff60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:05.335099 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7794e8-6d8f-4391-8666-41976104ff60 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:05.335385 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7794e8-6d8f-4391-8666-41976104ff60 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:05.335694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7794e8-6d8f-4391-8666-41976104ff60 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:05.336443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7794e8-6d8f-4391-8666-41976104ff60 None admin] RBAC: Adding query filter params (name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:05.336750 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7794e8-6d8f-4391-8666-41976104ff60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:05.338134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7794e8-6d8f-4391-8666-41976104ff60 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:05.346431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 284/561] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:31:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:05.347068 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7794e8-6d8f-4391-8666-41976104ff60 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:05.358708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 278/562] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:31:05.359434 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ef22d-8da9-4e8a-8049-79f71b5d1144 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:05.372323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:05.405422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:05.462183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7794e8-6d8f-4391-8666-41976104ff60 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:05.470476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ef22d-8da9-4e8a-8049-79f71b5d1144 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:05.495856 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76153615-a607-45b9-a9f4-1d8a39894b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:05.496692 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76153615-a607-45b9-a9f4-1d8a39894b59 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:05.496977 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76153615-a607-45b9-a9f4-1d8a39894b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:05.497251 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76153615-a607-45b9-a9f4-1d8a39894b59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:05.506910 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f4d3a-8627-42eb-bd46-198bf899c2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:05.507097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76153615-a607-45b9-a9f4-1d8a39894b59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:05.507950 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f4d3a-8627-42eb-bd46-198bf899c2ba None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:05.508402 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f4d3a-8627-42eb-bd46-198bf899c2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:05.508819 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f4d3a-8627-42eb-bd46-198bf899c2ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:05.509798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f4d3a-8627-42eb-bd46-198bf899c2ba None admin] RBAC: Adding query filter params (role.id=bb94781ab8054924a5cb21b561b12bbb, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=61b61188b5b94a51ba1ae1a88b78b89f) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:05.510264 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f4d3a-8627-42eb-bd46-198bf899c2ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:05.511966 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f4d3a-8627-42eb-bd46-198bf899c2ba None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:05.514641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-76153615-a607-45b9-a9f4-1d8a39894b59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:05.514641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:05.514641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:05.514641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:05.514641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:05.514641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:05.514641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:05.514641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:05.514641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:05.514641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:05.514641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:05.514641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:05.514641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:05.514641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:05.514641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:05.514641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:05.515777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:05.515777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:05.515777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:05.515777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:05.515777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:05.515777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:05.515777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:05.515777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:05.515777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:05.515777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:05.515777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:05.515777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:05.515777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:05.515777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:05.515777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:05.515777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:05.516899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:05.516899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:05.516899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:05.516899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:05.516899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:05.516899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:05.516899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:05.516899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:05.516899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:05.516899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:05.516899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:05.516899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:05.520682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 285/563] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:31:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:05.521039 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 279/564] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:05 2022] GET /identity/v3/role_assignments?user.id=61b61188b5b94a51ba1ae1a88b78b89f&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=bb94781ab8054924a5cb21b561b12bbb => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:05.521580 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76153615-a607-45b9-a9f4-1d8a39894b59 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:05.522110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f4d3a-8627-42eb-bd46-198bf899c2ba None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:05.617439 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:05.646350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f4d3a-8627-42eb-bd46-198bf899c2ba None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:05.676958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76153615-a607-45b9-a9f4-1d8a39894b59 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:05.680848 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:05.692836 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80536da7-0d40-46b6-bd83-d779633ffd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:05.693769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80536da7-0d40-46b6-bd83-d779633ffd53 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:05.694114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80536da7-0d40-46b6-bd83-d779633ffd53 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:05.694472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80536da7-0d40-46b6-bd83-d779633ffd53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:05.706079 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80536da7-0d40-46b6-bd83-d779633ffd53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:05.713998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-80536da7-0d40-46b6-bd83-d779633ffd53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:05.713998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:05.713998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:05.713998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:05.713998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:05.713998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:05.713998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:05.713998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:05.713998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:05.713998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:05.713998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:05.713998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:05.713998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:05.713998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:05.713998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:05.713998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:05.714983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:05.714983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:05.714983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:05.714983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:05.714983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:05.714983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:05.714983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:05.714983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:05.714983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:05.714983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:05.714983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:05.714983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:05.714983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:05.714983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:05.714983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:05.714983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:05.715870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:05.715870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:05.715870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:05.715870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:05.715870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:05.715870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:05.715870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:05.715870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:05.715870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:05.715870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:05.715870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:05.715870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:05.716506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 280/565] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:31:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:05.717102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80536da7-0d40-46b6-bd83-d779633ffd53 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:05.727564 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3f092-6716-4f12-ac2e-6386a5a4198e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:05.728671 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3f092-6716-4f12-ac2e-6386a5a4198e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:05.729171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3f092-6716-4f12-ac2e-6386a5a4198e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:05.729660 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3f092-6716-4f12-ac2e-6386a5a4198e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:05.741969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b3f092-6716-4f12-ac2e-6386a5a4198e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:05.743909 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b3f092-6716-4f12-ac2e-6386a5a4198e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:05.762292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-96b3f092-6716-4f12-ac2e-6386a5a4198e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:05.762292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:05.762292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:05.762292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:05.762292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:05.762292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:05.762292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:05.762292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:05.762292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:05.762292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:05.762292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:05.762292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:05.762292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:05.762292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:05.762292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:05.763438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:05.763438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:05.763438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:05.763438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:05.763438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:05.763438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:05.763438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:05.763438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:05.763438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:05.763438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:05.763438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:05.763438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:05.763438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:05.763438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:05.763438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:05.763438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:05.763438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:05.764355 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:05.764355 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:05.764355 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:05.764355 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:05.766185 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 286/566] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:05.767390 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3f092-6716-4f12-ac2e-6386a5a4198e None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:05.827219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80536da7-0d40-46b6-bd83-d779633ffd53 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:05.861900 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c72d1-2a0b-44be-9efe-2c11d9ec399e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:05.862716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c72d1-2a0b-44be-9efe-2c11d9ec399e None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:05.863004 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c72d1-2a0b-44be-9efe-2c11d9ec399e None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:05.863320 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c72d1-2a0b-44be-9efe-2c11d9ec399e None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:05.864097 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c72d1-2a0b-44be-9efe-2c11d9ec399e None admin] RBAC: Adding query filter params (name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:05.864379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c72d1-2a0b-44be-9efe-2c11d9ec399e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:05.865783 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c72d1-2a0b-44be-9efe-2c11d9ec399e None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:05.867386 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:05.875441 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 281/567] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:31:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:05.875817 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b3f092-6716-4f12-ac2e-6386a5a4198e None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:05.876192 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c72d1-2a0b-44be-9efe-2c11d9ec399e None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:05.913161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b3f73-d659-4a30-bf6d-c79c24f5c05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:05.914051 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b3f73-d659-4a30-bf6d-c79c24f5c05e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:05.914340 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b3f73-d659-4a30-bf6d-c79c24f5c05e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:05.914620 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b3f73-d659-4a30-bf6d-c79c24f5c05e None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:05.915442 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b3f73-d659-4a30-bf6d-c79c24f5c05e None admin] RBAC: Adding query filter params (name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:05.915755 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b3f73-d659-4a30-bf6d-c79c24f5c05e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:05.917538 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b3f73-d659-4a30-bf6d-c79c24f5c05e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:05.930145 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 287/568] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:31:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:05.930145 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5e269-5181-4840-a640-9287b4b39494 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:05.930145 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5e269-5181-4840-a640-9287b4b39494 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:05.930145 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5e269-5181-4840-a640-9287b4b39494 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:05.930145 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 288/569] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:05.930145 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5e269-5181-4840-a640-9287b4b39494 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:05.948726 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:05.988874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c72d1-2a0b-44be-9efe-2c11d9ec399e None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:06.024862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b914e-30bb-427e-96e0-a6e3023d009f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:06.025782 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b914e-30bb-427e-96e0-a6e3023d009f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:06.026249 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b914e-30bb-427e-96e0-a6e3023d009f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:06.026525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b914e-30bb-427e-96e0-a6e3023d009f None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:06.027293 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b914e-30bb-427e-96e0-a6e3023d009f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:06.027595 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b914e-30bb-427e-96e0-a6e3023d009f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:06.028987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b914e-30bb-427e-96e0-a6e3023d009f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:06.035931 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5e269-5181-4840-a640-9287b4b39494 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:06.041446 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 282/570] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:06.042464 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b914e-30bb-427e-96e0-a6e3023d009f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:06.075440 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa7339-3b1d-4e01-81dc-b4d6407191d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:06.076255 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa7339-3b1d-4e01-81dc-b4d6407191d2 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:06.076535 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa7339-3b1d-4e01-81dc-b4d6407191d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:06.076809 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa7339-3b1d-4e01-81dc-b4d6407191d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:06.087891 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa7339-3b1d-4e01-81dc-b4d6407191d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:06.089666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa7339-3b1d-4e01-81dc-b4d6407191d2 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:06.101891 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:06.107634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-f2aa7339-3b1d-4e01-81dc-b4d6407191d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:06.107634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:06.107634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:06.107634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:06.107634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:06.107634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:06.107634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:06.107634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:06.107634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:06.107634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:06.107634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:06.107634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:06.107634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:06.107634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:06.107634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:06.108561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:06.108561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:06.108561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:06.108561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:06.108561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:06.108561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:06.108561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:06.108561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:06.108561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:06.108561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:06.108561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:06.108561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:06.108561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:06.108561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:06.108561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:06.108561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:06.108561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:06.109450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:06.109450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:06.109450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:06.109450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:06.109450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 289/571] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:06.110067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3e9e8-6481-4668-bd28-ebd74a324004 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:06.110319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3e9e8-6481-4668-bd28-ebd74a324004 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:06.110599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea3e9e8-6481-4668-bd28-ebd74a324004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:06.127449 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-aea3e9e8-6481-4668-bd28-ebd74a324004 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:06.128967 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-aea3e9e8-6481-4668-bd28-ebd74a324004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:06.153912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b914e-30bb-427e-96e0-a6e3023d009f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:06.190569 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c661e53-1e81-453e-bbaa-9ec935e0f392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:06.191421 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c661e53-1e81-453e-bbaa-9ec935e0f392 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:06.191710 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c661e53-1e81-453e-bbaa-9ec935e0f392 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:06.191984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c661e53-1e81-453e-bbaa-9ec935e0f392 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:06.195185 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:06.200795 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 290/572] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:31:06.201965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3e9e8-6481-4668-bd28-ebd74a324004 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:06.209885 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c661e53-1e81-453e-bbaa-9ec935e0f392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:06.212028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c661e53-1e81-453e-bbaa-9ec935e0f392 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:06.232661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-4c661e53-1e81-453e-bbaa-9ec935e0f392 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:06.232661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:06.232661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:06.232661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:06.232661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:06.232661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:06.232661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:06.232661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:06.232661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:06.232661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:06.232661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:06.232661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:06.232661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:06.232661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:06.232661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:06.233775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:06.233775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:06.233775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:06.233775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:06.233775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:06.233775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:06.233775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:06.233775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:06.233775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:06.233775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:06.233775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:06.233775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:06.233775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:06.233775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:06.233775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:06.233775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:06.233775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:06.234787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:06.234787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:06.234787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:06.234787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:06.234787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 283/573] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:06.234787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c661e53-1e81-453e-bbaa-9ec935e0f392 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:06.315215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea3e9e8-6481-4668-bd28-ebd74a324004 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:06.352985 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3f1f6-c1a8-4cfe-a872-60fa8e886286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:06.354066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3f1f6-c1a8-4cfe-a872-60fa8e886286 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:06.354504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3f1f6-c1a8-4cfe-a872-60fa8e886286 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:06.354770 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3f1f6-c1a8-4cfe-a872-60fa8e886286 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:06.356137 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c661e53-1e81-453e-bbaa-9ec935e0f392 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:06.363544 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3f1f6-c1a8-4cfe-a872-60fa8e886286 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:06.365102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:06.365356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3f1f6-c1a8-4cfe-a872-60fa8e886286 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:06.382841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-89b3f1f6-c1a8-4cfe-a872-60fa8e886286 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:31:06.382841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:06.382841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:06.382841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:06.382841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:06.382841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:06.382841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:06.382841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:06.382841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:06.382841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:06.382841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:06.382841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:06.382841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:06.382841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:06.382841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:06.382841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:06.384019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:06.385157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:06.385157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:06.385157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:06.385157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:06.385157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:06.385157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:06.385157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:06.385157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:06.385157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:31:06.385157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:06.386138 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 291/574] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jul 21 05:31:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:06.386890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3f1f6-c1a8-4cfe-a872-60fa8e886286 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:06.398506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aaf236-d065-4209-ae8c-b8569b579438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:06.399560 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaf236-d065-4209-ae8c-b8569b579438 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:06.399850 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaf236-d065-4209-ae8c-b8569b579438 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:06.400128 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaf236-d065-4209-ae8c-b8569b579438 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:06.400882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aaf236-d065-4209-ae8c-b8569b579438 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:06.401471 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aaf236-d065-4209-ae8c-b8569b579438 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:06.402888 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aaf236-d065-4209-ae8c-b8569b579438 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:06.415388 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 284/575] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:06.416194 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aaf236-d065-4209-ae8c-b8569b579438 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:06.458374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:06.508052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3f1f6-c1a8-4cfe-a872-60fa8e886286 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:06.520775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aaf236-d065-4209-ae8c-b8569b579438 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:06.546605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28643b8-e933-4ae5-bd76-f3179b6479a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:06.548128 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28643b8-e933-4ae5-bd76-f3179b6479a7 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:06.548840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28643b8-e933-4ae5-bd76-f3179b6479a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:06.549292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28643b8-e933-4ae5-bd76-f3179b6479a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:06.557956 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303551aa-ee35-4064-96be-9f507e176f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:06.558927 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303551aa-ee35-4064-96be-9f507e176f54 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:06.559254 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303551aa-ee35-4064-96be-9f507e176f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:06.559528 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303551aa-ee35-4064-96be-9f507e176f54 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:06.560292 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303551aa-ee35-4064-96be-9f507e176f54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:06.560547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303551aa-ee35-4064-96be-9f507e176f54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:06.561011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28643b8-e933-4ae5-bd76-f3179b6479a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:06.561937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303551aa-ee35-4064-96be-9f507e176f54 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:06.568788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-b28643b8-e933-4ae5-bd76-f3179b6479a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:06.568788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:06.568788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:06.568788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:06.568788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:06.568788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:06.568788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:06.568788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:06.568788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:06.568788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:06.568788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:06.568788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:06.568788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:06.568788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:06.568788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:06.568788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:06.569786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:06.569786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:06.569786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:06.569786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:06.569786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:06.569786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:06.569786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:06.569786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:06.569786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:06.569786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:06.569786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:06.569786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:06.569786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:06.569786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:06.569786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:06.569786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:06.570674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:06.570674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:06.570674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:06.570674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:06.570674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:06.570674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:06.570674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:06.570674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:06.570674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:06.570674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:06.570674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:06.570674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:06.570674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 292/576] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:31:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:06.571389 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28643b8-e933-4ae5-bd76-f3179b6479a7 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:06.578286 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 285/577] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:06.579348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303551aa-ee35-4064-96be-9f507e176f54 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:06.611511 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:06.688663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303551aa-ee35-4064-96be-9f507e176f54 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:06.693593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28643b8-e933-4ae5-bd76-f3179b6479a7 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:06.714958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:06.722431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7c03e-1325-42a0-9f33-a16d98a9e655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:06.723255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7c03e-1325-42a0-9f33-a16d98a9e655 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:06.723535 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7c03e-1325-42a0-9f33-a16d98a9e655 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:06.723804 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7c03e-1325-42a0-9f33-a16d98a9e655 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:06.731278 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c2f58-bd97-42a5-beeb-c5a16c75effd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:06.731961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c2f58-bd97-42a5-beeb-c5a16c75effd None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:06.732239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7c03e-1325-42a0-9f33-a16d98a9e655 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:06.732339 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c2f58-bd97-42a5-beeb-c5a16c75effd None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:06.732511 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c2f58-bd97-42a5-beeb-c5a16c75effd None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:06.733348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c2f58-bd97-42a5-beeb-c5a16c75effd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:06.733659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c2f58-bd97-42a5-beeb-c5a16c75effd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:06.733751 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7c03e-1325-42a0-9f33-a16d98a9e655 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:06.735452 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c2f58-bd97-42a5-beeb-c5a16c75effd None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:06.747713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-42a7c03e-1325-42a0-9f33-a16d98a9e655 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 05:31:06.747713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:06.747713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:06.747713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:06.747713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:06.747713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:06.747713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:06.747713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:06.747713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:06.747713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:06.747713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:06.747713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:06.747713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:06.747713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:06.747713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:06.747713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:06.748670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:06.749598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:06.749598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:06.749598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:06.749598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:06.749598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:06.749598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:06.749598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:06.749598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:06.749598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 05:31:06.749598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:06.753712 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 286/578] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:31:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:06.754563 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 293/579] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jul 21 05:31:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:06.754704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7c03e-1325-42a0-9f33-a16d98a9e655 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:06.755590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c2f58-bd97-42a5-beeb-c5a16c75effd None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:06.844352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:06.857673 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7c03e-1325-42a0-9f33-a16d98a9e655 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:06.860151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c2f58-bd97-42a5-beeb-c5a16c75effd None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:06.891206 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad08b0-3ea4-4681-8019-95ef5e95d28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:06.891878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad08b0-3ea4-4681-8019-95ef5e95d28d None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:06.892158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad08b0-3ea4-4681-8019-95ef5e95d28d None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:06.892406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad08b0-3ea4-4681-8019-95ef5e95d28d None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:06.893163 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad08b0-3ea4-4681-8019-95ef5e95d28d None admin] RBAC: Adding query filter params (name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:06.893463 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad08b0-3ea4-4681-8019-95ef5e95d28d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:06.894833 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bad08b0-3ea4-4681-8019-95ef5e95d28d None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:06.895284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9ee77-b720-4a36-9eed-b9baca8a9fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:06.896200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9ee77-b720-4a36-9eed-b9baca8a9fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:06.896485 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9ee77-b720-4a36-9eed-b9baca8a9fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:06.896853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c9ee77-b720-4a36-9eed-b9baca8a9fe5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:06.903067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 287/580] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:31:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:06.903874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad08b0-3ea4-4681-8019-95ef5e95d28d None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:06.909094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9ee77-b720-4a36-9eed-b9baca8a9fe5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:06.910566 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c9ee77-b720-4a36-9eed-b9baca8a9fe5 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:06.926632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-73c9ee77-b720-4a36-9eed-b9baca8a9fe5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 05:31:06.926632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:06.926632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:06.926632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:06.926632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:06.926632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:06.926632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:06.926632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:06.926632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:06.926632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:06.926632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:06.926632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:06.926632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:06.926632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:06.926632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:06.926632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:06.928230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:06.929164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:06.929164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:06.929164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:06.929164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:06.929164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:06.929164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:06.929164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:06.929164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:06.929164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 05:31:06.929164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:06.929164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 294/581] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:31:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:06.929164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9ee77-b720-4a36-9eed-b9baca8a9fe5 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:06.971368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:07.006583 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad08b0-3ea4-4681-8019-95ef5e95d28d None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:07.032569 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c9ee77-b720-4a36-9eed-b9baca8a9fe5 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:07.045885 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45710d0b-4534-46c9-94c5-30abe5d00ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:07.046561 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45710d0b-4534-46c9-94c5-30abe5d00ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:07.046841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45710d0b-4534-46c9-94c5-30abe5d00ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:07.047119 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45710d0b-4534-46c9-94c5-30abe5d00ba3 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:07.047976 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45710d0b-4534-46c9-94c5-30abe5d00ba3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:07.048255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45710d0b-4534-46c9-94c5-30abe5d00ba3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:07.049684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45710d0b-4534-46c9-94c5-30abe5d00ba3 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:07.066982 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 288/582] 10.222.0.44 () {56 vars in 1137 bytes} [Thu Jul 21 05:31:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:07.067812 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45710d0b-4534-46c9-94c5-30abe5d00ba3 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:07.070141 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662887c1-74cf-450e-a739-c40f72e02673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:07.071167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662887c1-74cf-450e-a739-c40f72e02673 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:07.071454 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662887c1-74cf-450e-a739-c40f72e02673 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:07.071736 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662887c1-74cf-450e-a739-c40f72e02673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:07.079704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:07.082935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662887c1-74cf-450e-a739-c40f72e02673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:07.084586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662887c1-74cf-450e-a739-c40f72e02673 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:07.102079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-662887c1-74cf-450e-a739-c40f72e02673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:07.102079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:07.102079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:07.102079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:07.102079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:07.102079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:07.102079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:07.102079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:07.102079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:07.102079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:07.102079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:07.102079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:07.102079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:07.102079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:07.102079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:07.103067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:07.103067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:07.103067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:07.103067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:07.103067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:07.103067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:07.103067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:07.103067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:07.103067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:07.103067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:07.103067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:07.103067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:07.103067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:07.103067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:07.103067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:07.103067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:07.103067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:07.104157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:07.104157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:07.104157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:07.104157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:07.108125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 295/583] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:07.108970 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662887c1-74cf-450e-a739-c40f72e02673 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:07.173786 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45710d0b-4534-46c9-94c5-30abe5d00ba3 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:07.196180 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:07.207284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662887c1-74cf-450e-a739-c40f72e02673 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:07.210267 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80faafaa-f1f2-4649-9558-e5cd6288f78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:07.210405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faafaa-f1f2-4649-9558-e5cd6288f78d None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:07.210749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faafaa-f1f2-4649-9558-e5cd6288f78d None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:07.211042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80faafaa-f1f2-4649-9558-e5cd6288f78d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:07.221744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80faafaa-f1f2-4649-9558-e5cd6288f78d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:07.223221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80faafaa-f1f2-4649-9558-e5cd6288f78d None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:07.240828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-80faafaa-f1f2-4649-9558-e5cd6288f78d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:07.240828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:07.240828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:07.240828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:07.240828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:07.240828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:07.240828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:07.240828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:07.240828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:07.240828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:07.240828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:07.240828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:07.240828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:07.240828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:07.240828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:07.241813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:07.241813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:07.241813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:07.241813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:07.241813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:07.241813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:07.241813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:07.241813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:07.241813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:07.241813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:07.241813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:07.241813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:07.241813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:07.241813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:07.241813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:07.241813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:07.241813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:07.242703 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:07.242703 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:07.242703 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:07.242703 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:07.242703 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 289/584] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:07.242998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80faafaa-f1f2-4649-9558-e5cd6288f78d None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:07.251463 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22b0f8-e258-4d99-8ef9-9d9511f028ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:07.252151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22b0f8-e258-4d99-8ef9-9d9511f028ef None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:07.252431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22b0f8-e258-4d99-8ef9-9d9511f028ef None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:07.252701 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22b0f8-e258-4d99-8ef9-9d9511f028ef None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:07.253537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22b0f8-e258-4d99-8ef9-9d9511f028ef None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:07.253838 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22b0f8-e258-4d99-8ef9-9d9511f028ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:07.255360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d22b0f8-e258-4d99-8ef9-9d9511f028ef None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:07.277583 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 296/585] 10.222.0.44 () {56 vars in 1137 bytes} [Thu Jul 21 05:31:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:07.277583 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22b0f8-e258-4d99-8ef9-9d9511f028ef None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:07.309614 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:07.352284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80faafaa-f1f2-4649-9558-e5cd6288f78d None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:07.383884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22b0f8-e258-4d99-8ef9-9d9511f028ef None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:07.388303 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b867c-cfc2-498d-8fd4-0026ebfa0ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:07.389107 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b867c-cfc2-498d-8fd4-0026ebfa0ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:07.389386 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b867c-cfc2-498d-8fd4-0026ebfa0ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:07.389657 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b867c-cfc2-498d-8fd4-0026ebfa0ef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:07.399953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b867c-cfc2-498d-8fd4-0026ebfa0ef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:07.401586 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b867c-cfc2-498d-8fd4-0026ebfa0ef2 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:07.419057 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d82255-c87c-41da-94bc-1d432c1195b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:07.419778 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d82255-c87c-41da-94bc-1d432c1195b8 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:07.420059 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d82255-c87c-41da-94bc-1d432c1195b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:07.420331 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d82255-c87c-41da-94bc-1d432c1195b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:07.420444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-d13b867c-cfc2-498d-8fd4-0026ebfa0ef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:07.420444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:07.420444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:07.420444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:07.420444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:07.420444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:07.420444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:07.420444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:07.420444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:07.420444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:07.420444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:07.420444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:07.420444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:07.420444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:07.420444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:07.421308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:07.421308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:07.421308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:07.421308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:07.421308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:07.421308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:07.421308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:07.421308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:07.421308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:07.421308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:07.421308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:07.421308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:07.421308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:07.421308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:07.421308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:07.421308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:07.421308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:07.422615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:07.422615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:07.422615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:07.422615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:07.422994 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d82255-c87c-41da-94bc-1d432c1195b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:07.422994 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d82255-c87c-41da-94bc-1d432c1195b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:07.423159 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 290/586] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:07.423159 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b867c-cfc2-498d-8fd4-0026ebfa0ef2 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:07.427389 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d82255-c87c-41da-94bc-1d432c1195b8 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:07.439496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 297/587] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:07.440292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d82255-c87c-41da-94bc-1d432c1195b8 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:07.452955 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:07.532306 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b867c-cfc2-498d-8fd4-0026ebfa0ef2 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:07.544766 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d82255-c87c-41da-94bc-1d432c1195b8 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:07.547589 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:07.569083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d6f7c-15e3-4124-ae3c-68d7eb1bf5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:07.569745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6f7c-15e3-4124-ae3c-68d7eb1bf5ce None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:07.570091 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6f7c-15e3-4124-ae3c-68d7eb1bf5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:07.570389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d6f7c-15e3-4124-ae3c-68d7eb1bf5ce None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:07.571181 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d6f7c-15e3-4124-ae3c-68d7eb1bf5ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:07.571495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d6f7c-15e3-4124-ae3c-68d7eb1bf5ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:07.572900 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d6f7c-15e3-4124-ae3c-68d7eb1bf5ce None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:07.582568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135aff0a-6e91-4e12-8607-b9397c8111de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:07.583566 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135aff0a-6e91-4e12-8607-b9397c8111de None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:07.583847 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135aff0a-6e91-4e12-8607-b9397c8111de None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:07.584120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135aff0a-6e91-4e12-8607-b9397c8111de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:07.584598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 291/588] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:07.585282 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d6f7c-15e3-4124-ae3c-68d7eb1bf5ce None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:07.601035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135aff0a-6e91-4e12-8607-b9397c8111de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:07.602738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135aff0a-6e91-4e12-8607-b9397c8111de None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:07.619552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-135aff0a-6e91-4e12-8607-b9397c8111de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:07.619552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:07.619552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:07.619552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:07.619552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:07.619552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:07.619552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:07.619552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:07.619552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:07.619552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:07.619552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:07.619552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:07.619552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:07.619552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:07.619552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:07.620458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:07.620458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:07.620458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:07.620458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:07.620458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:07.620458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:07.620458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:07.620458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:07.620458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:07.620458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:07.620458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:07.620458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:07.620458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:07.620458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:07.620458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:07.620458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:07.620458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:07.621422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:07.621422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:07.621422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:07.621422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:07.621422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 298/589] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:07.621703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135aff0a-6e91-4e12-8607-b9397c8111de None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:07.678824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:07.685570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d6f7c-15e3-4124-ae3c-68d7eb1bf5ce None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:07.723355 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135aff0a-6e91-4e12-8607-b9397c8111de None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:07.723547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80286e-c8d8-4b13-9ac0-2e21d7773a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:07.723947 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80286e-c8d8-4b13-9ac0-2e21d7773a18 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:07.724227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80286e-c8d8-4b13-9ac0-2e21d7773a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:07.724496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80286e-c8d8-4b13-9ac0-2e21d7773a18 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:07.725214 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80286e-c8d8-4b13-9ac0-2e21d7773a18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:07.725527 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80286e-c8d8-4b13-9ac0-2e21d7773a18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:07.727055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80286e-c8d8-4b13-9ac0-2e21d7773a18 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:07.738523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 292/590] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:07.739279 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80286e-c8d8-4b13-9ac0-2e21d7773a18 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:07.761169 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87470be7-ceba-4b64-883a-ebb43238e2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:07.761975 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87470be7-ceba-4b64-883a-ebb43238e2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:07.762254 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87470be7-ceba-4b64-883a-ebb43238e2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:07.762529 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87470be7-ceba-4b64-883a-ebb43238e2a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:07.769115 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:07.769310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87470be7-ceba-4b64-883a-ebb43238e2a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:07.771073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87470be7-ceba-4b64-883a-ebb43238e2a3 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:07.785073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-87470be7-ceba-4b64-883a-ebb43238e2a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:07.785073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:07.785073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:07.785073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:07.785073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:07.785073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:07.785073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:07.785073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:07.785073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:07.785073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:07.785073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:07.785073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:07.785073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:07.785073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:07.785073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:07.786134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:07.786134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:07.786134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:07.786134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:07.786134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:07.786134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:07.786134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:07.786134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:07.786134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:07.786134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:07.786134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:07.786134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:07.786134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:07.786134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:07.786134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:07.786134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:07.786134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:07.792407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:07.792407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:07.792407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:07.792407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:07.792407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:07.792407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:07.792407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:07.792407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:07.792407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 299/591] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:07.792407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56595ad-665c-4fed-bc7d-75bbc4b4a536 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:07.792407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56595ad-665c-4fed-bc7d-75bbc4b4a536 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:07.793012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56595ad-665c-4fed-bc7d-75bbc4b4a536 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:07.793012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 300/592] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:07.793012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56595ad-665c-4fed-bc7d-75bbc4b4a536 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:07.839875 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80286e-c8d8-4b13-9ac0-2e21d7773a18 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:07.874888 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9e66b-03d0-4667-986e-aed2381e7563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:07.875727 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9e66b-03d0-4667-986e-aed2381e7563 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:07.876019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9e66b-03d0-4667-986e-aed2381e7563 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:07.876299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9e66b-03d0-4667-986e-aed2381e7563 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:07.883568 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c9e66b-03d0-4667-986e-aed2381e7563 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:07.885070 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c9e66b-03d0-4667-986e-aed2381e7563 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:07.890370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56595ad-665c-4fed-bc7d-75bbc4b4a536 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:07.899220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-97c9e66b-03d0-4667-986e-aed2381e7563 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 05:31:07.899220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:07.899220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:07.899220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:07.899220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:07.899220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:07.899220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:07.899220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:07.899220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:07.899220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:07.899220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:07.899220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:07.899220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:07.899220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:07.899220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:07.899220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:07.900255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:07.906195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:07.906195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:07.906195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:07.906195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:07.906195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:07.906195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:07.906195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:07.906195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:07.906195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 05:31:07.906195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:07.906195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 293/593] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:31:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:07.906195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9e66b-03d0-4667-986e-aed2381e7563 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:07.911618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:07.926418 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9161c8-09a7-4b9e-8ba2-95ab428cebeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:07.927111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9161c8-09a7-4b9e-8ba2-95ab428cebeb None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:07.927646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9161c8-09a7-4b9e-8ba2-95ab428cebeb None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:07.927646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9161c8-09a7-4b9e-8ba2-95ab428cebeb None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:07.928356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9161c8-09a7-4b9e-8ba2-95ab428cebeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:07.928663 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9161c8-09a7-4b9e-8ba2-95ab428cebeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:07.930099 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9161c8-09a7-4b9e-8ba2-95ab428cebeb None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:07.940973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 301/594] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:07.941749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9161c8-09a7-4b9e-8ba2-95ab428cebeb None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:07.990277 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:07.999552 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9e66b-03d0-4667-986e-aed2381e7563 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:08.034436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca6779-91ed-4635-bbc5-d26608552429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:08.035264 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca6779-91ed-4635-bbc5-d26608552429 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:08.035558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca6779-91ed-4635-bbc5-d26608552429 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:08.035834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca6779-91ed-4635-bbc5-d26608552429 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:08.043403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca6779-91ed-4635-bbc5-d26608552429 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:08.044313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9161c8-09a7-4b9e-8ba2-95ab428cebeb None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:08.044855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca6779-91ed-4635-bbc5-d26608552429 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:08.059121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-b9ca6779-91ed-4635-bbc5-d26608552429 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:08.059121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:08.059121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:08.059121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:08.059121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:08.059121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:08.059121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:08.059121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:08.059121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:08.059121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:08.059121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:08.059121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:08.059121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:08.059121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:08.059121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:08.060054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:08.060054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:08.060054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:08.060054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:08.060054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:08.060054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:08.060054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:08.060054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:08.060054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:08.060054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:08.060054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:08.060054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:08.060054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:08.060054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:08.060054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:08.060054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:08.060054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:08.060943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:08.060943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:08.060943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:08.060943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:08.060943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:08.060943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:08.060943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:08.060943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:08.060943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 294/595] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:08.061755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41da1d5-356c-4854-91fe-c2a54bf335a0 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:08.062043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41da1d5-356c-4854-91fe-c2a54bf335a0 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:08.062320 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41da1d5-356c-4854-91fe-c2a54bf335a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:08.078188 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-e41da1d5-356c-4854-91fe-c2a54bf335a0 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:08.079371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e41da1d5-356c-4854-91fe-c2a54bf335a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:08.079935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517e889-4f4d-4f85-930c-7ecb98640021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:08.080611 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517e889-4f4d-4f85-930c-7ecb98640021 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:08.080893 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517e889-4f4d-4f85-930c-7ecb98640021 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:08.081170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517e889-4f4d-4f85-930c-7ecb98640021 None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:08.081959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517e889-4f4d-4f85-930c-7ecb98640021 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:08.082266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517e889-4f4d-4f85-930c-7ecb98640021 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:08.083884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517e889-4f4d-4f85-930c-7ecb98640021 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:08.096142 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 302/596] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:08.096928 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517e889-4f4d-4f85-930c-7ecb98640021 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:08.135618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:08.148657 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 295/597] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:31:08.149499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41da1d5-356c-4854-91fe-c2a54bf335a0 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:08.194670 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517e889-4f4d-4f85-930c-7ecb98640021 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:08.228303 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2c6b4-6c47-41eb-ab97-532441215546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:08.228982 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2c6b4-6c47-41eb-ab97-532441215546 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:08.229272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2c6b4-6c47-41eb-ab97-532441215546 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:08.229530 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2c6b4-6c47-41eb-ab97-532441215546 None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:08.229671 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:08.230363 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2c6b4-6c47-41eb-ab97-532441215546 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:08.230815 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2c6b4-6c47-41eb-ab97-532441215546 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:08.232471 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2c6b4-6c47-41eb-ab97-532441215546 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:08.247882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41da1d5-356c-4854-91fe-c2a54bf335a0 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:08.249591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 303/598] 10.222.0.44 () {56 vars in 1139 bytes} [Thu Jul 21 05:31:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:08.250391 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2c6b4-6c47-41eb-ab97-532441215546 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:08.288437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d544bc1-7f66-4cb0-ae54-5ab77ce35777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:08.289335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d544bc1-7f66-4cb0-ae54-5ab77ce35777 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:08.289617 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d544bc1-7f66-4cb0-ae54-5ab77ce35777 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:08.289887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d544bc1-7f66-4cb0-ae54-5ab77ce35777 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:08.296652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d544bc1-7f66-4cb0-ae54-5ab77ce35777 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:08.298128 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d544bc1-7f66-4cb0-ae54-5ab77ce35777 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:08.311409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-4d544bc1-7f66-4cb0-ae54-5ab77ce35777 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:08.311409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:08.311409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:08.311409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:08.311409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:08.311409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:08.311409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:08.311409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:08.311409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:08.311409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:08.311409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:08.311409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:08.311409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:08.311409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:08.311409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:08.312472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:08.312472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:08.312472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:08.312472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:08.312472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:08.312472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:08.312472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:08.312472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:08.312472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:08.312472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:08.312472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:08.312472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:08.312472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:08.312472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:08.312472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:08.312472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:08.312472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:08.313420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:08.313420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:08.313420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:08.313420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:08.313420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:08.313420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:08.313420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:08.313420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:08.313420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 296/599] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:08.313930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d544bc1-7f66-4cb0-ae54-5ab77ce35777 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:08.355195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:08.359270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2c6b4-6c47-41eb-ab97-532441215546 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:08.397202 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a560b3-11fd-4667-bd92-1c393cb33dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:08.397874 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a560b3-11fd-4667-bd92-1c393cb33dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:08.398161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a560b3-11fd-4667-bd92-1c393cb33dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:08.398439 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a560b3-11fd-4667-bd92-1c393cb33dd1 None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:08.399259 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a560b3-11fd-4667-bd92-1c393cb33dd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:08.399571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a560b3-11fd-4667-bd92-1c393cb33dd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:08.401012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a560b3-11fd-4667-bd92-1c393cb33dd1 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:08.413519 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 304/600] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:08.414351 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a560b3-11fd-4667-bd92-1c393cb33dd1 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:08.428477 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d544bc1-7f66-4cb0-ae54-5ab77ce35777 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:08.461248 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abfd03-891a-493d-99ba-d317b6eb3fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:08.462159 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abfd03-891a-493d-99ba-d317b6eb3fe0 None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:08.462475 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abfd03-891a-493d-99ba-d317b6eb3fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:08.462763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abfd03-891a-493d-99ba-d317b6eb3fe0 None admin] PATH_INFO: `/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/928d014648cd41d7b58e7e01845eb748/roles/bb94781ab8054924a5cb21b561b12bbb` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:08.474119 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:08.477253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abfd03-891a-493d-99ba-d317b6eb3fe0 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b6f27b8d9b44d191210a24a9e04041, user_id=928d014648cd41d7b58e7e01845eb748, role_id=bb94781ab8054924a5cb21b561b12bbb)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:08.479070 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abfd03-891a-493d-99ba-d317b6eb3fe0 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:08.492777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 297/601] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:31:08 2022] PUT /identity/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/928d014648cd41d7b58e7e01845eb748/roles/bb94781ab8054924a5cb21b561b12bbb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:31:08.493560 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abfd03-891a-493d-99ba-d317b6eb3fe0 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:08.529326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a560b3-11fd-4667-bd92-1c393cb33dd1 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:08.569313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92499336-e4a5-40b1-8950-8d780588b114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:08.570359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92499336-e4a5-40b1-8950-8d780588b114 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:08.570644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92499336-e4a5-40b1-8950-8d780588b114 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:08.570917 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92499336-e4a5-40b1-8950-8d780588b114 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:08.584554 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92499336-e4a5-40b1-8950-8d780588b114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:08.593158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-92499336-e4a5-40b1-8950-8d780588b114 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:08.593158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:08.593158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:08.593158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:08.593158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:08.593158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:08.593158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:08.593158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:08.593158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:08.593158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:08.593158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:08.593158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:08.593158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:08.593158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:08.593158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:08.593158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:08.594210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:08.594210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:08.594210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:08.594210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:08.594210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:08.594210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:08.594210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:08.594210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:08.594210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:08.594210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:08.594210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:08.594210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:08.594210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:08.594210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:08.594210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:08.594210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:08.595097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:08.595097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:08.595097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:08.595097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:08.595097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:08.595097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:08.595097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:08.595097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:08.595097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:08.595097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:08.595097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:08.595097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:08.599568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 305/602] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:31:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:08.600596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92499336-e4a5-40b1-8950-8d780588b114 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:08.614745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abfd03-891a-493d-99ba-d317b6eb3fe0 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:08.622064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:08.664659 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d6977-3c40-4dd9-bd9b-5cec9b4ad911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:08.665491 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d6977-3c40-4dd9-bd9b-5cec9b4ad911 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:08.665772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d6977-3c40-4dd9-bd9b-5cec9b4ad911 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:08.666047 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d6977-3c40-4dd9-bd9b-5cec9b4ad911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:08.678375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d6977-3c40-4dd9-bd9b-5cec9b4ad911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:08.679955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d6977-3c40-4dd9-bd9b-5cec9b4ad911 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:08.698485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-4d5d6977-3c40-4dd9-bd9b-5cec9b4ad911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:08.698485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:08.698485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:08.698485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:08.698485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:08.698485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:08.698485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:08.698485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:08.698485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:08.698485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:08.698485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:08.698485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:08.698485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:08.698485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:08.698485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:08.699608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:08.699608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:08.699608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:08.699608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:08.699608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:08.699608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:08.699608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:08.699608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:08.699608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:08.699608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:08.699608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:08.699608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:08.699608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:08.699608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:08.699608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:08.699608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:08.699608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:08.700533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:08.700533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:08.700533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:08.700533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:08.700533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 298/603] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:08.700823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d6977-3c40-4dd9-bd9b-5cec9b4ad911 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:08.732022 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92499336-e4a5-40b1-8950-8d780588b114 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:08.751335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:08.766428 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093d17f-eb0d-453d-aa00-342322117d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:08.767159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093d17f-eb0d-453d-aa00-342322117d76 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:08.767448 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093d17f-eb0d-453d-aa00-342322117d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:08.767824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093d17f-eb0d-453d-aa00-342322117d76 None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:08.768507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093d17f-eb0d-453d-aa00-342322117d76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:08.768816 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093d17f-eb0d-453d-aa00-342322117d76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:08.770259 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093d17f-eb0d-453d-aa00-342322117d76 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:08.788034 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 306/604] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:08.788034 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093d17f-eb0d-453d-aa00-342322117d76 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:08.816046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d6977-3c40-4dd9-bd9b-5cec9b4ad911 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:08.851588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8991e2-ce76-4b15-a5c0-c354258f932a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:08.852488 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8991e2-ce76-4b15-a5c0-c354258f932a None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:08.852772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8991e2-ce76-4b15-a5c0-c354258f932a None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:08.853049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8991e2-ce76-4b15-a5c0-c354258f932a None admin] PATH_INFO: `/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/2707c6967a0a419a98ae142ff1d1a961/roles/bb94781ab8054924a5cb21b561b12bbb` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:08.867831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:08.867990 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8991e2-ce76-4b15-a5c0-c354258f932a None admin] RBAC: Authorizing `identity:create_grant(project_id=74b6f27b8d9b44d191210a24a9e04041, user_id=2707c6967a0a419a98ae142ff1d1a961, role_id=bb94781ab8054924a5cb21b561b12bbb)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:08.869430 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8991e2-ce76-4b15-a5c0-c354258f932a None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:08.883739 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 299/605] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:31:08 2022] PUT /identity/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/2707c6967a0a419a98ae142ff1d1a961/roles/bb94781ab8054924a5cb21b561b12bbb => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:31:08.884613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8991e2-ce76-4b15-a5c0-c354258f932a None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:08.900142 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093d17f-eb0d-453d-aa00-342322117d76 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:08.942425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91346d7-0058-462a-96d5-b1edce492a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:08.943936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91346d7-0058-462a-96d5-b1edce492a7f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:08.944427 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91346d7-0058-462a-96d5-b1edce492a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:08.944914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91346d7-0058-462a-96d5-b1edce492a7f None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:08.945931 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91346d7-0058-462a-96d5-b1edce492a7f None admin] RBAC: Adding query filter params (name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:08.946402 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91346d7-0058-462a-96d5-b1edce492a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:08.948153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91346d7-0058-462a-96d5-b1edce492a7f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:08.959055 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 307/606] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:31:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:08.965332 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91346d7-0058-462a-96d5-b1edce492a7f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:08.990354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:09.000509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8991e2-ce76-4b15-a5c0-c354258f932a None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:09.047232 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171cfa1-0160-4d19-a71b-b98c11519453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:09.048715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171cfa1-0160-4d19-a71b-b98c11519453 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:09.049186 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171cfa1-0160-4d19-a71b-b98c11519453 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:09.049651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171cfa1-0160-4d19-a71b-b98c11519453 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:09.050727 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e171cfa1-0160-4d19-a71b-b98c11519453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:09.051245 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e171cfa1-0160-4d19-a71b-b98c11519453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:09.052963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e171cfa1-0160-4d19-a71b-b98c11519453 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:09.071871 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 300/607] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:09.071871 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171cfa1-0160-4d19-a71b-b98c11519453 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:09.095678 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91346d7-0058-462a-96d5-b1edce492a7f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:09.138135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00e4a6-eeed-4000-898c-e2170b0e15b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:09.139325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00e4a6-eeed-4000-898c-e2170b0e15b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:09.139855 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00e4a6-eeed-4000-898c-e2170b0e15b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:09.140135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00e4a6-eeed-4000-898c-e2170b0e15b8 None admin] PATH_INFO: `/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/f3f76371c6a04fc5a477d6ef1908a08e/roles/bb94781ab8054924a5cb21b561b12bbb` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:09.152783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:09.156989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00e4a6-eeed-4000-898c-e2170b0e15b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b6f27b8d9b44d191210a24a9e04041, user_id=f3f76371c6a04fc5a477d6ef1908a08e, role_id=bb94781ab8054924a5cb21b561b12bbb)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:09.158582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00e4a6-eeed-4000-898c-e2170b0e15b8 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:09.177776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 308/608] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:31:08 2022] PUT /identity/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/f3f76371c6a04fc5a477d6ef1908a08e/roles/bb94781ab8054924a5cb21b561b12bbb => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:31:09.179207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00e4a6-eeed-4000-898c-e2170b0e15b8 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:09.187441 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171cfa1-0160-4d19-a71b-b98c11519453 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:09.238756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a97e0-10c9-4a69-9109-c5c7f1b03b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:09.239621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a97e0-10c9-4a69-9109-c5c7f1b03b30 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:09.239901 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a97e0-10c9-4a69-9109-c5c7f1b03b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:09.240177 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a97e0-10c9-4a69-9109-c5c7f1b03b30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:09.269952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a97e0-10c9-4a69-9109-c5c7f1b03b30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:09.271489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a97e0-10c9-4a69-9109-c5c7f1b03b30 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:09.304049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-a42a97e0-10c9-4a69-9109-c5c7f1b03b30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:09.304049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:09.304049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:09.304049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:09.304049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:09.304049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:09.304049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:09.304049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:09.304049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:09.304049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:09.304049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:09.304049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:09.304049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:09.304049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:09.304049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:09.305148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:09.305148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:09.305148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:09.305148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:09.305148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:09.305148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:09.305148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:09.305148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:09.305148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:09.305148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:09.305148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:09.305148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:09.305148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:09.305148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:09.305148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:09.305148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:09.305148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:09.306055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:09.306055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:09.306055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:09.306055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:09.316672 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 301/609] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:09.317762 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a97e0-10c9-4a69-9109-c5c7f1b03b30 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:09.330324 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:09.357471 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00e4a6-eeed-4000-898c-e2170b0e15b8 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:09.394431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea113a71-b759-484c-a539-ca4739492e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:09.395274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea113a71-b759-484c-a539-ca4739492e4f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:09.395558 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea113a71-b759-484c-a539-ca4739492e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:09.395837 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea113a71-b759-484c-a539-ca4739492e4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:09.403289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea113a71-b759-484c-a539-ca4739492e4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:09.404828 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea113a71-b759-484c-a539-ca4739492e4f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:09.422513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-ea113a71-b759-484c-a539-ca4739492e4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:09.422513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:09.422513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:09.422513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:09.422513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:09.422513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:09.422513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:09.422513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:09.422513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:09.422513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:09.422513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:09.422513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:09.422513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:09.422513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:09.422513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:09.424150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:09.424150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:09.424150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:09.424150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:09.424150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:09.424150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:09.424150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:09.424150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:09.424150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:09.424150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:09.424150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:09.424150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:09.424150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:09.424150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:09.424150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:09.424150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:09.424150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:09.425804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:09.425804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:09.425804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:09.425804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:09.425804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:09.425804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:09.425804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:09.425804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:09.425804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 309/610] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:09.440164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea113a71-b759-484c-a539-ca4739492e4f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:09.455464 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a97e0-10c9-4a69-9109-c5c7f1b03b30 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:09.493762 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6ea99-f26f-4969-90cf-7db0b6f2b47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:09.494432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6ea99-f26f-4969-90cf-7db0b6f2b47a None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:09.494711 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6ea99-f26f-4969-90cf-7db0b6f2b47a None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:09.494983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6ea99-f26f-4969-90cf-7db0b6f2b47a None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:09.495768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6ea99-f26f-4969-90cf-7db0b6f2b47a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:09.496070 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6ea99-f26f-4969-90cf-7db0b6f2b47a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:09.497558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6ea99-f26f-4969-90cf-7db0b6f2b47a None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:09.510555 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 302/611] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:09.516123 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6ea99-f26f-4969-90cf-7db0b6f2b47a None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:09.535407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:09.567615 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea113a71-b759-484c-a539-ca4739492e4f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:09.594525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:09.630218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6ea99-f26f-4969-90cf-7db0b6f2b47a None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:09.644310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b99f62-2457-4db8-a0e0-61e2d3f88844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:09.645148 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b99f62-2457-4db8-a0e0-61e2d3f88844 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:09.645262 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b99f62-2457-4db8-a0e0-61e2d3f88844 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:09.645543 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b99f62-2457-4db8-a0e0-61e2d3f88844 None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:09.646427 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b99f62-2457-4db8-a0e0-61e2d3f88844 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:09.646656 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b99f62-2457-4db8-a0e0-61e2d3f88844 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:09.648244 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b99f62-2457-4db8-a0e0-61e2d3f88844 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:09.671883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1096e94-bee0-46b2-9042-26ad7819fe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:09.672934 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1096e94-bee0-46b2-9042-26ad7819fe6f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:09.673358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1096e94-bee0-46b2-9042-26ad7819fe6f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:09.673780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1096e94-bee0-46b2-9042-26ad7819fe6f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:09.681813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1096e94-bee0-46b2-9042-26ad7819fe6f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:09.683537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1096e94-bee0-46b2-9042-26ad7819fe6f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:09.689458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 310/612] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:09.699622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-b1096e94-bee0-46b2-9042-26ad7819fe6f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 05:31:09.699622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:09.699622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:09.699622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:09.699622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:09.699622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:09.699622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:09.699622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:09.699622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:09.699622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:09.699622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:09.699622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:09.699622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:09.699622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:09.699622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:09.699622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:09.700763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:09.701942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:09.701942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:09.701942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:09.701942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:09.701942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:09.701942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:09.701942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:09.701942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:09.701942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 05:31:09.701942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:09.702605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b99f62-2457-4db8-a0e0-61e2d3f88844 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:09.704352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 303/613] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:31:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:09.710573 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1096e94-bee0-46b2-9042-26ad7819fe6f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:09.815043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1096e94-bee0-46b2-9042-26ad7819fe6f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:09.849156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:09.859647 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45239c34-4967-4db3-b139-13143b9accfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:09.860524 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45239c34-4967-4db3-b139-13143b9accfe None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:09.860974 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45239c34-4967-4db3-b139-13143b9accfe None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:09.861398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45239c34-4967-4db3-b139-13143b9accfe None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:09.862394 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45239c34-4967-4db3-b139-13143b9accfe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:09.862840 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45239c34-4967-4db3-b139-13143b9accfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:09.864559 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45239c34-4967-4db3-b139-13143b9accfe None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:09.885927 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 304/614] 10.222.0.44 () {56 vars in 1135 bytes} [Thu Jul 21 05:31:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:09.891657 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45239c34-4967-4db3-b139-13143b9accfe None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:09.933106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b99f62-2457-4db8-a0e0-61e2d3f88844 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:09.951410 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:09.992862 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b995985-93c4-4405-bfe1-152ba5c454e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:09.993752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b995985-93c4-4405-bfe1-152ba5c454e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:09.994036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b995985-93c4-4405-bfe1-152ba5c454e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:09.994306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b995985-93c4-4405-bfe1-152ba5c454e5 None admin] PATH_INFO: `/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/e07508a9fa9146528620b7f8abec8549/roles/bb94781ab8054924a5cb21b561b12bbb` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:10.023976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b995985-93c4-4405-bfe1-152ba5c454e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b6f27b8d9b44d191210a24a9e04041, user_id=e07508a9fa9146528620b7f8abec8549, role_id=bb94781ab8054924a5cb21b561b12bbb)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:10.025459 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b995985-93c4-4405-bfe1-152ba5c454e5 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:10.046288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 311/615] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:31:09 2022] PUT /identity/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/e07508a9fa9146528620b7f8abec8549/roles/bb94781ab8054924a5cb21b561b12bbb => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:31:10.047456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45239c34-4967-4db3-b139-13143b9accfe None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:10.085691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a8ba3-5a6f-4215-92fa-d01cb7564d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:10.086499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a8ba3-5a6f-4215-92fa-d01cb7564d14 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:10.086780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a8ba3-5a6f-4215-92fa-d01cb7564d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:10.087091 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a8ba3-5a6f-4215-92fa-d01cb7564d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:10.098841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a8ba3-5a6f-4215-92fa-d01cb7564d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:10.100552 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a8ba3-5a6f-4215-92fa-d01cb7564d14 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:10.119676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-fc2a8ba3-5a6f-4215-92fa-d01cb7564d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:10.119676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:10.119676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:10.119676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:10.119676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:10.119676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:10.119676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:10.119676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:10.119676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:10.119676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:10.119676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:10.119676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:10.119676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:10.119676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:10.119676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:10.120662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:10.120662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:10.120662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:10.120662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:10.120662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:10.120662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:10.120662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:10.120662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:10.120662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:10.120662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:10.120662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:10.120662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:10.120662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:10.120662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:10.120662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:10.120662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:10.120662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:10.121547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:10.121547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:10.121547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:10.121547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:10.121547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 305/616] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:10.128146 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b995985-93c4-4405-bfe1-152ba5c454e5 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:10.255517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b995985-93c4-4405-bfe1-152ba5c454e5 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:10.305215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8e832-896d-476e-8a92-9d7336f835d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:10.305953 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8e832-896d-476e-8a92-9d7336f835d8 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:10.306281 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8e832-896d-476e-8a92-9d7336f835d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:10.306570 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8e832-896d-476e-8a92-9d7336f835d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:10.307326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8e832-896d-476e-8a92-9d7336f835d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:10.307629 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8e832-896d-476e-8a92-9d7336f835d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:10.309051 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8e832-896d-476e-8a92-9d7336f835d8 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:10.326536 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 312/617] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:10.332656 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a8ba3-5a6f-4215-92fa-d01cb7564d14 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:10.374104 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:10.447417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a8ba3-5a6f-4215-92fa-d01cb7564d14 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:10.489145 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b37f0-8bdf-482a-9523-87db409a0eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:10.489968 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b37f0-8bdf-482a-9523-87db409a0eed None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:10.490251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b37f0-8bdf-482a-9523-87db409a0eed None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:10.490526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b37f0-8bdf-482a-9523-87db409a0eed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:10.497959 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b37f0-8bdf-482a-9523-87db409a0eed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:10.499467 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b37f0-8bdf-482a-9523-87db409a0eed None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:10.520760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-906b37f0-8bdf-482a-9523-87db409a0eed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:10.520760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:10.520760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:10.520760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:10.520760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:10.520760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:10.520760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:10.520760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:10.520760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:10.520760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:10.520760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:10.520760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:10.520760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:10.520760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:10.520760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:10.521662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:10.521662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:10.521662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:10.521662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:10.521662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:10.521662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:10.521662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:10.521662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:10.521662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:10.521662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:10.521662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:10.521662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:10.521662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:10.521662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:10.521662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:10.521662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:10.521662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:10.522702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:10.522702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:10.522702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:10.522702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:10.522702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:10.522702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:10.522702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:10.522702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:10.522702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 306/618] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:10.532280 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8e832-896d-476e-8a92-9d7336f835d8 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:10.570094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:10.677300 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8e832-896d-476e-8a92-9d7336f835d8 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:10.720104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a668d30-7066-482b-8d67-7f21797de339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:10.720841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a668d30-7066-482b-8d67-7f21797de339 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:10.721123 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a668d30-7066-482b-8d67-7f21797de339 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:10.721396 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a668d30-7066-482b-8d67-7f21797de339 None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:10.722237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a668d30-7066-482b-8d67-7f21797de339 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:10.722554 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a668d30-7066-482b-8d67-7f21797de339 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:10.724018 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a668d30-7066-482b-8d67-7f21797de339 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:10.736993 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 313/619] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:10.742907 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b37f0-8bdf-482a-9523-87db409a0eed None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:10.783340 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0c0c6-0235-46d9-bc97-1cae3f72fa6e None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:10.783677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0c0c6-0235-46d9-bc97-1cae3f72fa6e None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:10.783876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0c0c6-0235-46d9-bc97-1cae3f72fa6e None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:10.785521 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 314/620] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:10.790906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf641898-cbb0-4bda-b856-c2808e82cb7c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:10.791238 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf641898-cbb0-4bda-b856-c2808e82cb7c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:10.791518 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf641898-cbb0-4bda-b856-c2808e82cb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:10.808130 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-bf641898-cbb0-4bda-b856-c2808e82cb7c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:10.809316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-bf641898-cbb0-4bda-b856-c2808e82cb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:10.917814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b37f0-8bdf-482a-9523-87db409a0eed None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:10.926835 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:10.961734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 315/621] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:31:10.973646 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271f079-7122-4023-a0ce-e0d4fb31f27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:10.974871 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271f079-7122-4023-a0ce-e0d4fb31f27d None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:10.975412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271f079-7122-4023-a0ce-e0d4fb31f27d None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:10.975877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271f079-7122-4023-a0ce-e0d4fb31f27d None admin] PATH_INFO: `/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/61b61188b5b94a51ba1ae1a88b78b89f/roles/bb94781ab8054924a5cb21b561b12bbb` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:10.990989 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271f079-7122-4023-a0ce-e0d4fb31f27d None admin] RBAC: Authorizing `identity:create_grant(project_id=74b6f27b8d9b44d191210a24a9e04041, user_id=61b61188b5b94a51ba1ae1a88b78b89f, role_id=bb94781ab8054924a5cb21b561b12bbb)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:10.992999 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271f079-7122-4023-a0ce-e0d4fb31f27d None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:10.995766 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf641898-cbb0-4bda-b856-c2808e82cb7c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:11.012284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:11.033532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 307/622] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:31:10 2022] PUT /identity/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/61b61188b5b94a51ba1ae1a88b78b89f/roles/bb94781ab8054924a5cb21b561b12bbb => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:31:11.132522 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf641898-cbb0-4bda-b856-c2808e82cb7c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:11.169639 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c557f9-9fee-4f03-a161-731f5c4a4ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:11.170819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c557f9-9fee-4f03-a161-731f5c4a4ecd None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:11.171320 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c557f9-9fee-4f03-a161-731f5c4a4ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:11.171801 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c557f9-9fee-4f03-a161-731f5c4a4ecd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:11.183345 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c557f9-9fee-4f03-a161-731f5c4a4ecd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:11.191550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-d9c557f9-9fee-4f03-a161-731f5c4a4ecd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:11.191550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:11.191550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:11.191550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:11.191550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:11.191550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:11.191550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:11.191550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:11.191550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:11.191550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:11.191550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:11.191550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:11.191550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:11.191550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:11.191550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:11.191550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:11.192497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:11.192497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:11.192497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:11.192497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:11.192497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:11.192497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:11.192497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:11.192497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:11.192497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:11.192497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:11.192497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:11.192497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:11.192497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:11.192497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:11.192497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:11.192497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:11.193356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:11.193356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:11.193356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:11.193356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:11.193356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:11.193356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:11.193356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:11.193356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:11.193356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:11.193356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:11.193356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:11.193356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:11.196359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 316/623] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:31:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:11.204509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271f079-7122-4023-a0ce-e0d4fb31f27d None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:11.230696 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b590b6-9977-40fc-9f8e-965483a5d888 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:11.231078 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b590b6-9977-40fc-9f8e-965483a5d888 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:11.231559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b590b6-9977-40fc-9f8e-965483a5d888 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:11.233301 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 317/624] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:31:11.238821 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc407abd-69f7-4b63-838d-04ed69ea2a23 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:11.239142 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc407abd-69f7-4b63-838d-04ed69ea2a23 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:11.239611 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc407abd-69f7-4b63-838d-04ed69ea2a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:11.258856 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-bc407abd-69f7-4b63-838d-04ed69ea2a23 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:11.260120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-bc407abd-69f7-4b63-838d-04ed69ea2a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:11.376139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271f079-7122-4023-a0ce-e0d4fb31f27d None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:11.382169 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:11.391686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 318/625] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:31:11.416158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a126b6d-f622-4c02-afb5-f9d4f97b9e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:11.416845 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a126b6d-f622-4c02-afb5-f9d4f97b9e1f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:11.417131 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a126b6d-f622-4c02-afb5-f9d4f97b9e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:11.417408 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a126b6d-f622-4c02-afb5-f9d4f97b9e1f None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:11.418167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a126b6d-f622-4c02-afb5-f9d4f97b9e1f None admin] RBAC: Adding query filter params (name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:11.418501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a126b6d-f622-4c02-afb5-f9d4f97b9e1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:11.420052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a126b6d-f622-4c02-afb5-f9d4f97b9e1f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:11.422101 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc407abd-69f7-4b63-838d-04ed69ea2a23 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:11.434001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 308/626] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:31:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:11.441555 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a126b6d-f622-4c02-afb5-f9d4f97b9e1f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:11.462815 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:11.535302 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc407abd-69f7-4b63-838d-04ed69ea2a23 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:11.571211 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992daa4c-6ad1-40b8-8c8a-2d15902e63e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:11.572031 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992daa4c-6ad1-40b8-8c8a-2d15902e63e2 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:11.572307 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992daa4c-6ad1-40b8-8c8a-2d15902e63e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:11.572583 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992daa4c-6ad1-40b8-8c8a-2d15902e63e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:11.583219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992daa4c-6ad1-40b8-8c8a-2d15902e63e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:11.591152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-992daa4c-6ad1-40b8-8c8a-2d15902e63e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:11.591152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:11.591152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:11.591152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:11.591152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:11.591152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:11.591152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:11.591152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:11.591152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:11.591152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:11.591152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:11.591152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:11.591152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:11.591152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:11.591152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:11.591152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:11.592149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:11.592149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:11.592149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:11.592149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:11.592149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:11.592149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:11.592149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:11.592149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:11.592149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:11.592149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:11.592149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:11.592149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:11.592149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:11.592149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:11.592149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:11.592149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:11.593274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:11.593274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:11.593274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:11.593274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:11.593274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:11.593274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:11.593274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:11.593274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:11.593274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:11.593274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:11.593274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:11.593274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:11.593274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 319/627] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:31:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:11.601274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a126b6d-f622-4c02-afb5-f9d4f97b9e1f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:11.604024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992daa4c-6ad1-40b8-8c8a-2d15902e63e2 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:11.655821 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04d3b7-fc6e-4399-9df5-6504fea1ec2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:11.657474 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04d3b7-fc6e-4399-9df5-6504fea1ec2a None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:11.657970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04d3b7-fc6e-4399-9df5-6504fea1ec2a None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:11.658563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04d3b7-fc6e-4399-9df5-6504fea1ec2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:11.670490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:11.680383 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04d3b7-fc6e-4399-9df5-6504fea1ec2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:11.681869 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04d3b7-fc6e-4399-9df5-6504fea1ec2a None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:11.712284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-2f04d3b7-fc6e-4399-9df5-6504fea1ec2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:11.712284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:11.712284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:11.712284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:11.712284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:11.712284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:11.712284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:11.712284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:11.712284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:11.712284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:11.712284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:11.712284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:11.712284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:11.712284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:11.712284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:11.713297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:11.713297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:11.713297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:11.713297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:11.713297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:11.713297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:11.713297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:11.713297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:11.713297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:11.713297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:11.713297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:11.713297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:11.713297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:11.713297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:11.713297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:11.713297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:11.713297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:11.714299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:11.714299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:11.714299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:11.714299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:11.714528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992daa4c-6ad1-40b8-8c8a-2d15902e63e2 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:11.714613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 309/628] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:11.720611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04d3b7-fc6e-4399-9df5-6504fea1ec2a None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:11.778990 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e2cb7-7cf8-47ed-bdfb-f82ce8b1707f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:11.779708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e2cb7-7cf8-47ed-bdfb-f82ce8b1707f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:11.779990 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e2cb7-7cf8-47ed-bdfb-f82ce8b1707f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:11.780265 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e2cb7-7cf8-47ed-bdfb-f82ce8b1707f None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:11.781010 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e2cb7-7cf8-47ed-bdfb-f82ce8b1707f None admin] RBAC: Adding query filter params (name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:11.781316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e2cb7-7cf8-47ed-bdfb-f82ce8b1707f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:11.782711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e2cb7-7cf8-47ed-bdfb-f82ce8b1707f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:11.796822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 320/629] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:31:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:11.802751 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e2cb7-7cf8-47ed-bdfb-f82ce8b1707f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:11.817011 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:11.876253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04d3b7-fc6e-4399-9df5-6504fea1ec2a None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:11.913965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcfaac2-602f-403e-9075-23ad5876da57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:11.915055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfaac2-602f-403e-9075-23ad5876da57 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:11.915570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfaac2-602f-403e-9075-23ad5876da57 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:11.916023 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcfaac2-602f-403e-9075-23ad5876da57 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:11.917071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcfaac2-602f-403e-9075-23ad5876da57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:11.917604 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcfaac2-602f-403e-9075-23ad5876da57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:11.919137 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcfaac2-602f-403e-9075-23ad5876da57 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:11.934550 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 310/630] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:11.936012 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e02fe1-e515-427a-803e-07f43af65d80 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:11.936369 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e02fe1-e515-427a-803e-07f43af65d80 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:11.936740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e02fe1-e515-427a-803e-07f43af65d80 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:11.941016 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e2cb7-7cf8-47ed-bdfb-f82ce8b1707f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:11.942512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 311/631] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:11.943491 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e02fe1-e515-427a-803e-07f43af65d80 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:11.968987 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:11.984689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13b799-9024-43c5-82f9-9e39707f8745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:11.985746 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13b799-9024-43c5-82f9-9e39707f8745 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:11.986206 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13b799-9024-43c5-82f9-9e39707f8745 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:11.986657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13b799-9024-43c5-82f9-9e39707f8745 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:12.000797 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13b799-9024-43c5-82f9-9e39707f8745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:12.002515 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13b799-9024-43c5-82f9-9e39707f8745 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:12.023686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-2e13b799-9024-43c5-82f9-9e39707f8745 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:12.023686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:12.023686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:12.023686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:12.023686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:12.023686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:12.023686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:12.023686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:12.023686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:12.023686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:12.023686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:12.023686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:12.023686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:12.023686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:12.023686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:12.024634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:12.024634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:12.024634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:12.024634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:12.024634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:12.024634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:12.024634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:12.024634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:12.024634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:12.024634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:12.024634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:12.024634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:12.024634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:12.024634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:12.024634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:12.024634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:12.024634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:12.025544 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:12.025544 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:12.025544 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:12.025544 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:12.025544 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 321/632] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:12.026375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc2bac-0295-4f3d-8b2c-b3fd7e0c7b4c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:12.026763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc2bac-0295-4f3d-8b2c-b3fd7e0c7b4c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:12.027161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc2bac-0295-4f3d-8b2c-b3fd7e0c7b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:12.043762 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-90dc2bac-0295-4f3d-8b2c-b3fd7e0c7b4c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:12.045080 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-90dc2bac-0295-4f3d-8b2c-b3fd7e0c7b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:12.060471 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e02fe1-e515-427a-803e-07f43af65d80 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:12.084186 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:12.101364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae967d77-9c1c-4958-809a-264e3ccfc504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:12.102490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae967d77-9c1c-4958-809a-264e3ccfc504 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:12.102912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae967d77-9c1c-4958-809a-264e3ccfc504 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:12.103379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae967d77-9c1c-4958-809a-264e3ccfc504 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:12.111579 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae967d77-9c1c-4958-809a-264e3ccfc504 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:12.113200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae967d77-9c1c-4958-809a-264e3ccfc504 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:12.115095 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 322/633] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:31:12.116185 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc2bac-0295-4f3d-8b2c-b3fd7e0c7b4c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:12.141955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-ae967d77-9c1c-4958-809a-264e3ccfc504 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:31:12.141955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:12.141955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:12.141955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:12.141955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:12.141955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:12.141955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:12.141955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:12.141955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:12.141955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:12.141955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:12.141955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:12.141955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:12.141955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:12.141955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:12.141955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:12.142997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:12.143987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:12.143987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:12.143987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:12.143987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:12.143987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:12.143987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:12.143987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:12.143987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:12.143987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:31:12.143987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:12.146354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 312/634] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jul 21 05:31:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:12.147366 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae967d77-9c1c-4958-809a-264e3ccfc504 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:12.250966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc2bac-0295-4f3d-8b2c-b3fd7e0c7b4c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:12.252857 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae967d77-9c1c-4958-809a-264e3ccfc504 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:12.253325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:12.288016 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4bbb6-0964-4802-9d33-c7327802b7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:12.288807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4bbb6-0964-4802-9d33-c7327802b7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:12.289035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4bbb6-0964-4802-9d33-c7327802b7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:12.289312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4bbb6-0964-4802-9d33-c7327802b7f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:12.290036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4bbb6-0964-4802-9d33-c7327802b7f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:12.290342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4bbb6-0964-4802-9d33-c7327802b7f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:12.291758 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a4bbb6-0964-4802-9d33-c7327802b7f8 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:12.292472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db8102-8fa1-4d6d-bf2b-b14eba6c7ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:12.293449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db8102-8fa1-4d6d-bf2b-b14eba6c7ded None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:12.293807 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db8102-8fa1-4d6d-bf2b-b14eba6c7ded None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:12.294231 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db8102-8fa1-4d6d-bf2b-b14eba6c7ded None admin] PATH_INFO: `/v3/roles/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:12.303999 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 323/635] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:12.304987 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4bbb6-0964-4802-9d33-c7327802b7f8 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:12.305625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db8102-8fa1-4d6d-bf2b-b14eba6c7ded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:12.313625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-a9db8102-8fa1-4d6d-bf2b-b14eba6c7ded None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:12.313625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:12.313625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:12.313625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:12.313625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:12.313625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:12.313625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:12.313625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:12.313625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:12.313625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:12.313625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:12.313625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:12.313625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:12.313625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:12.313625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:12.313625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:12.314615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:12.314615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:12.314615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:12.314615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:12.314615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:12.314615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:12.314615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:12.314615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:12.314615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:12.314615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:12.314615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:12.314615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:12.314615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:12.314615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:12.314615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:12.314615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:12.316344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:12.316344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:12.316344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:12.316344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:12.316344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:12.316344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:12.316344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:12.316344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:12.316344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:12.316344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:12.316344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:12.316344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:12.316344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 313/636] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:31:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:12.319979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db8102-8fa1-4d6d-bf2b-b14eba6c7ded None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:12.366204 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:12.407676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4bbb6-0964-4802-9d33-c7327802b7f8 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:12.433237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db8102-8fa1-4d6d-bf2b-b14eba6c7ded None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:12.442894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac097c86-1221-4cb5-9667-3788cec95adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:12.443812 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac097c86-1221-4cb5-9667-3788cec95adc None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:12.444242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac097c86-1221-4cb5-9667-3788cec95adc None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:12.444686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac097c86-1221-4cb5-9667-3788cec95adc None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:12.445676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac097c86-1221-4cb5-9667-3788cec95adc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:12.446127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac097c86-1221-4cb5-9667-3788cec95adc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:12.447748 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac097c86-1221-4cb5-9667-3788cec95adc None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:12.469712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 324/637] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jul 21 05:31:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:12.469905 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1b0ce-f40c-4e57-a3b3-365f22ddabfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:12.470574 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1b0ce-f40c-4e57-a3b3-365f22ddabfb None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:12.470860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1b0ce-f40c-4e57-a3b3-365f22ddabfb None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:12.471172 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1b0ce-f40c-4e57-a3b3-365f22ddabfb None admin] PATH_INFO: `/v3/users/glance` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:12.471432 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac097c86-1221-4cb5-9667-3788cec95adc None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:12.479564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1b0ce-f40c-4e57-a3b3-365f22ddabfb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:12.481143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1b0ce-f40c-4e57-a3b3-365f22ddabfb None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:12.496359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:12.497715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-a7a1b0ce-f40c-4e57-a3b3-365f22ddabfb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 21 05:31:12.497715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:12.497715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:12.497715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:12.497715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:12.497715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:12.497715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:12.497715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:12.497715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:12.497715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:12.497715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:12.497715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:12.497715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:12.497715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:12.497715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:12.497715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:12.498920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:12.500413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:12.500413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:12.500413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:12.500413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:12.500413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:12.500413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:12.500413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:12.500413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:12.500413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 21 05:31:12.500413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:12.500413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 314/638] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:31:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:12.500413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1b0ce-f40c-4e57-a3b3-365f22ddabfb None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:12.577861 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac097c86-1221-4cb5-9667-3788cec95adc None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:12.612746 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:12.613243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e190ef-1558-494c-ba78-544e2e1d85af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:12.614088 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e190ef-1558-494c-ba78-544e2e1d85af None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:12.614514 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e190ef-1558-494c-ba78-544e2e1d85af None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:12.614940 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e190ef-1558-494c-ba78-544e2e1d85af None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:12.615856 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e190ef-1558-494c-ba78-544e2e1d85af None admin] RBAC: Adding query filter params (name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:12.616319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e190ef-1558-494c-ba78-544e2e1d85af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:12.616433 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1b0ce-f40c-4e57-a3b3-365f22ddabfb None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:12.618028 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e190ef-1558-494c-ba78-544e2e1d85af None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:12.631568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 325/639] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:31:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:12.632742 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e190ef-1558-494c-ba78-544e2e1d85af None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:12.666272 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe4510c-11d9-4559-a0ad-7021c6d35401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:12.667170 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4510c-11d9-4559-a0ad-7021c6d35401 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:12.667603 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4510c-11d9-4559-a0ad-7021c6d35401 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:12.668125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4510c-11d9-4559-a0ad-7021c6d35401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:12.718987 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:12.734964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e190ef-1558-494c-ba78-544e2e1d85af None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:12.743441 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe4510c-11d9-4559-a0ad-7021c6d35401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:12.744935 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe4510c-11d9-4559-a0ad-7021c6d35401 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:12.769857 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181d78d-3fd4-4a6a-a2da-212e5dc827cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:12.770530 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181d78d-3fd4-4a6a-a2da-212e5dc827cb None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:12.770849 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181d78d-3fd4-4a6a-a2da-212e5dc827cb None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:12.771118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181d78d-3fd4-4a6a-a2da-212e5dc827cb None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:12.771954 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d181d78d-3fd4-4a6a-a2da-212e5dc827cb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:12.772262 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d181d78d-3fd4-4a6a-a2da-212e5dc827cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:12.773764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d181d78d-3fd4-4a6a-a2da-212e5dc827cb None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:12.791749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 326/640] 10.222.0.44 () {56 vars in 1137 bytes} [Thu Jul 21 05:31:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:12.792583 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181d78d-3fd4-4a6a-a2da-212e5dc827cb None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:12.848752 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:12.853702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 315/641] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:31:12.855007 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1762ff-0b46-4dec-bfa9-e1fb235c4a8d None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:12.855339 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1762ff-0b46-4dec-bfa9-e1fb235c4a8d None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:12.855619 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1762ff-0b46-4dec-bfa9-e1fb235c4a8d None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:12.860976 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 316/642] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:12.860976 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1762ff-0b46-4dec-bfa9-e1fb235c4a8d None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:12.903908 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181d78d-3fd4-4a6a-a2da-212e5dc827cb None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:12.940796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e2a38-df36-4d07-9321-1b91fd371552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:12.941600 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e2a38-df36-4d07-9321-1b91fd371552 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:12.941885 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e2a38-df36-4d07-9321-1b91fd371552 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:12.942161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e2a38-df36-4d07-9321-1b91fd371552 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:12.952461 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e2a38-df36-4d07-9321-1b91fd371552 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:12.953925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e2a38-df36-4d07-9321-1b91fd371552 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:12.956032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:12.972047 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1762ff-0b46-4dec-bfa9-e1fb235c4a8d None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:12.973066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-c15e2a38-df36-4d07-9321-1b91fd371552 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:12.973066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:12.973066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:12.973066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:12.973066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:12.973066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:12.973066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:12.973066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:12.973066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:12.973066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:12.973066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:12.973066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:12.973066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:12.973066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:12.973066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:12.974497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:12.974497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:12.974497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:12.974497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:12.974497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:12.974497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:12.974497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:12.974497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:12.974497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:12.974497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:12.974497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:12.974497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:12.974497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:12.974497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:12.974497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:12.974497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:12.974497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:12.975710 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:12.975710 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:12.975710 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:12.975710 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:12.975710 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 327/643] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:12.975710 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e2a38-df36-4d07-9321-1b91fd371552 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:13.015052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb04935-0eda-40d7-a594-b9daf0585d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:13.015713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb04935-0eda-40d7-a594-b9daf0585d2f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:13.015995 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb04935-0eda-40d7-a594-b9daf0585d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:13.016268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb04935-0eda-40d7-a594-b9daf0585d2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:13.080318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e2a38-df36-4d07-9321-1b91fd371552 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:13.089485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:13.092415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb04935-0eda-40d7-a594-b9daf0585d2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:13.094299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb04935-0eda-40d7-a594-b9daf0585d2f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:13.119946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f2ece-82a6-4fbe-9ea2-3a86798c4851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:13.120705 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f2ece-82a6-4fbe-9ea2-3a86798c4851 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:13.120987 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f2ece-82a6-4fbe-9ea2-3a86798c4851 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:13.121261 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f2ece-82a6-4fbe-9ea2-3a86798c4851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:13.131409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f2ece-82a6-4fbe-9ea2-3a86798c4851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:13.132938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f2ece-82a6-4fbe-9ea2-3a86798c4851 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:13.151338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-478f2ece-82a6-4fbe-9ea2-3a86798c4851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:13.151338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:13.151338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:13.151338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:13.151338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:13.151338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:13.151338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:13.151338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:13.151338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:13.151338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:13.151338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:13.151338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:13.151338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:13.151338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:13.151338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:13.152474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:13.152474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:13.152474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:13.152474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:13.152474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:13.152474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:13.152474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:13.152474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:13.152474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:13.152474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:13.152474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:13.152474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:13.152474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:13.152474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:13.152474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:13.152474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:13.152474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:13.153749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:13.153749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:13.153749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:13.153749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:13.157155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 328/644] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:13.158477 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c66f7-1953-4bee-9304-98c920b3cd24 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:13.158772 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c66f7-1953-4bee-9304-98c920b3cd24 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:13.159073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c66f7-1953-4bee-9304-98c920b3cd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:13.176601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-581c66f7-1953-4bee-9304-98c920b3cd24 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:13.177980 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-581c66f7-1953-4bee-9304-98c920b3cd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:13.202606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 317/645] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:31:13.203332 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb04935-0eda-40d7-a594-b9daf0585d2f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:13.226482 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:13.259210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 329/646] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:31:13.259210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c66f7-1953-4bee-9304-98c920b3cd24 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:13.311521 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:13.314646 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb04935-0eda-40d7-a594-b9daf0585d2f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:13.350804 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14042d-ecc1-4376-8a4f-a960c114b60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:13.351540 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14042d-ecc1-4376-8a4f-a960c114b60c None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:13.351832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14042d-ecc1-4376-8a4f-a960c114b60c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:13.352111 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a14042d-ecc1-4376-8a4f-a960c114b60c None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:13.352840 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14042d-ecc1-4376-8a4f-a960c114b60c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:13.353156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14042d-ecc1-4376-8a4f-a960c114b60c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:13.354561 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a14042d-ecc1-4376-8a4f-a960c114b60c None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:13.366335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 318/647] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:13.367669 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14042d-ecc1-4376-8a4f-a960c114b60c None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:13.376817 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c66f7-1953-4bee-9304-98c920b3cd24 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:13.415752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0a09e-4551-4729-aa95-dabef5dd7dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:13.416592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0a09e-4551-4729-aa95-dabef5dd7dbf None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:13.416879 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0a09e-4551-4729-aa95-dabef5dd7dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:13.417157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0a09e-4551-4729-aa95-dabef5dd7dbf None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:13.417880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0a09e-4551-4729-aa95-dabef5dd7dbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:13.418188 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0a09e-4551-4729-aa95-dabef5dd7dbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:13.419852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0a09e-4551-4729-aa95-dabef5dd7dbf None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:13.437598 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 330/648] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:13.437598 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0a09e-4551-4729-aa95-dabef5dd7dbf None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:13.469140 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:13.480357 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a14042d-ecc1-4376-8a4f-a960c114b60c None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:13.513203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450c24c-b546-4f8a-a6fa-36d36cdcc312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:13.514018 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450c24c-b546-4f8a-a6fa-36d36cdcc312 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:13.514274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450c24c-b546-4f8a-a6fa-36d36cdcc312 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:13.514539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450c24c-b546-4f8a-a6fa-36d36cdcc312 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:13.525129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450c24c-b546-4f8a-a6fa-36d36cdcc312 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:13.526582 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450c24c-b546-4f8a-a6fa-36d36cdcc312 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:13.532291 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0a09e-4551-4729-aa95-dabef5dd7dbf None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:13.544025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-8450c24c-b546-4f8a-a6fa-36d36cdcc312 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:13.544025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:13.544025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:13.544025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:13.544025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:13.544025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:13.544025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:13.544025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:13.544025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:13.544025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:13.544025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:13.544025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:13.544025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:13.544025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:13.544025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:13.544948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:13.544948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:13.544948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:13.544948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:13.544948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:13.544948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:13.544948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:13.544948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:13.544948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:13.544948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:13.544948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:13.544948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:13.544948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:13.544948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:13.544948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:13.544948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:13.544948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:13.545846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:13.545846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:13.545846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:13.545846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:13.545846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 319/649] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:13.546128 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450c24c-b546-4f8a-a6fa-36d36cdcc312 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:13.561557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:13.570763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cdad9d-4801-4524-b359-2965c7c0b2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:13.571612 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdad9d-4801-4524-b359-2965c7c0b2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:13.571897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdad9d-4801-4524-b359-2965c7c0b2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:13.572301 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdad9d-4801-4524-b359-2965c7c0b2e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:13.582232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cdad9d-4801-4524-b359-2965c7c0b2e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:13.589853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-a9cdad9d-4801-4524-b359-2965c7c0b2e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:13.589853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:13.589853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:13.589853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:13.589853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:13.589853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:13.589853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:13.589853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:13.589853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:13.589853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:13.589853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:13.589853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:13.589853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:13.589853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:13.589853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:13.589853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:13.591002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:13.591002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:13.591002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:13.591002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:13.591002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:13.591002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:13.591002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:13.591002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:13.591002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:13.591002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:13.591002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:13.591002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:13.591002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:13.591002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:13.591002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:13.591002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:13.591946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:13.591946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:13.591946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:13.591946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:13.591946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:13.591946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:13.591946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:13.591946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:13.591946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:13.591946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:13.591946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:13.591946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:13.595304 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 331/650] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:31:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:13.595996 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cdad9d-4801-4524-b359-2965c7c0b2e6 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:13.649579 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450c24c-b546-4f8a-a6fa-36d36cdcc312 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:13.683774 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7f332-d19c-4060-b92d-a85fb5d83a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:13.684577 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7f332-d19c-4060-b92d-a85fb5d83a2e None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:13.684856 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7f332-d19c-4060-b92d-a85fb5d83a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:13.685126 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7f332-d19c-4060-b92d-a85fb5d83a2e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:13.691863 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:13.693031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7f332-d19c-4060-b92d-a85fb5d83a2e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:13.694440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7f332-d19c-4060-b92d-a85fb5d83a2e None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:13.698976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cdad9d-4801-4524-b359-2965c7c0b2e6 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:13.708142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-b6a7f332-d19c-4060-b92d-a85fb5d83a2e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 05:31:13.708142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:13.708142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:13.708142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:13.708142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:13.708142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:13.708142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:13.708142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:13.708142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:13.708142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:13.708142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:13.708142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:13.708142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:13.708142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:13.708142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:13.708142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:13.709200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:13.710154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:13.710154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:13.710154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:13.710154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:13.710154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:13.710154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:13.710154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:13.710154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:13.710154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 05:31:13.710154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:13.710154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 320/651] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:31:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:13.710792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b4892-8378-4067-82e7-5157ba7908bf None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:13.710792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b4892-8378-4067-82e7-5157ba7908bf None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:13.711095 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b4892-8378-4067-82e7-5157ba7908bf None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:13.712309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 321/652] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:13.712985 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b4892-8378-4067-82e7-5157ba7908bf None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:13.738869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef3d62-a91b-40bc-9363-915b236b2aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:13.739703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef3d62-a91b-40bc-9363-915b236b2aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:13.739982 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef3d62-a91b-40bc-9363-915b236b2aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:13.740256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef3d62-a91b-40bc-9363-915b236b2aa5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:13.747547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ef3d62-a91b-40bc-9363-915b236b2aa5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:13.750048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ef3d62-a91b-40bc-9363-915b236b2aa5 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:13.770944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-b7ef3d62-a91b-40bc-9363-915b236b2aa5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:13.770944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:13.770944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:13.770944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:13.770944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:13.770944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:13.770944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:13.770944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:13.770944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:13.770944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:13.770944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:13.770944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:13.770944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:13.770944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:13.770944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:13.772054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:13.772054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:13.772054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:13.772054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:13.772054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:13.772054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:13.772054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:13.772054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:13.772054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:13.772054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:13.772054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:13.772054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:13.772054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:13.772054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:13.772054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:13.772054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:13.772054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:13.773007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:13.773007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:13.773007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:13.773007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:13.773007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:13.773007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:13.773007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:13.773007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:13.773007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 332/653] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:13.774005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef3d62-a91b-40bc-9363-915b236b2aa5 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:13.788380 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:13.819110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b4892-8378-4067-82e7-5157ba7908bf None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:13.852613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8bf22-4287-4d71-98fc-769204c09d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:13.853261 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8bf22-4287-4d71-98fc-769204c09d75 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:13.853541 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8bf22-4287-4d71-98fc-769204c09d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:13.853811 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8bf22-4287-4d71-98fc-769204c09d75 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:13.854529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8bf22-4287-4d71-98fc-769204c09d75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:13.854830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8bf22-4287-4d71-98fc-769204c09d75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:13.856228 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd8bf22-4287-4d71-98fc-769204c09d75 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:13.867445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 322/654] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:13.868223 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8bf22-4287-4d71-98fc-769204c09d75 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:13.871637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef3d62-a91b-40bc-9363-915b236b2aa5 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:13.908366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b11c64-c3c8-4144-ad9e-f57a0b47082d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:13.909037 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b11c64-c3c8-4144-ad9e-f57a0b47082d None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:13.909384 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b11c64-c3c8-4144-ad9e-f57a0b47082d None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:13.909717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b11c64-c3c8-4144-ad9e-f57a0b47082d None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:13.910524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b11c64-c3c8-4144-ad9e-f57a0b47082d None admin] RBAC: Adding query filter params (name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:13.910895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b11c64-c3c8-4144-ad9e-f57a0b47082d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:13.912351 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b11c64-c3c8-4144-ad9e-f57a0b47082d None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:13.920622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 333/655] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:31:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:13.921838 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecff67-7620-47ac-a9c6-ab79f1d6a541 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:13.922174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecff67-7620-47ac-a9c6-ab79f1d6a541 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:13.922504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecff67-7620-47ac-a9c6-ab79f1d6a541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:13.938155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-82ecff67-7620-47ac-a9c6-ab79f1d6a541 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:13.939349 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-82ecff67-7620-47ac-a9c6-ab79f1d6a541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:13.951852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:13.969359 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8bf22-4287-4d71-98fc-769204c09d75 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:13.999698 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:14.003728 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 334/656] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:31:14.007499 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecff67-7620-47ac-a9c6-ab79f1d6a541 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:14.008331 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61853e-3943-4809-81a6-5385b2714cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:14.009019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61853e-3943-4809-81a6-5385b2714cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:14.009303 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61853e-3943-4809-81a6-5385b2714cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:14.009577 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61853e-3943-4809-81a6-5385b2714cd0 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:14.010420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61853e-3943-4809-81a6-5385b2714cd0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:14.010730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61853e-3943-4809-81a6-5385b2714cd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:14.012184 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61853e-3943-4809-81a6-5385b2714cd0 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:14.030460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 323/657] 10.222.0.44 () {56 vars in 1137 bytes} [Thu Jul 21 05:31:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:14.031364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61853e-3943-4809-81a6-5385b2714cd0 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:14.115561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecff67-7620-47ac-a9c6-ab79f1d6a541 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:14.129359 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61853e-3943-4809-81a6-5385b2714cd0 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:14.148139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8171b77-bceb-4451-8a2f-95d354478c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:14.148801 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8171b77-bceb-4451-8a2f-95d354478c82 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:14.149106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8171b77-bceb-4451-8a2f-95d354478c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:14.149360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8171b77-bceb-4451-8a2f-95d354478c82 None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:14.150140 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8171b77-bceb-4451-8a2f-95d354478c82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:14.150446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8171b77-bceb-4451-8a2f-95d354478c82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:14.151925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8171b77-bceb-4451-8a2f-95d354478c82 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:14.162353 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f45a5-7891-4d4b-92a1-bf0d9e0b1147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:14.163470 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 335/658] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:14.163609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f45a5-7891-4d4b-92a1-bf0d9e0b1147 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:14.163806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f45a5-7891-4d4b-92a1-bf0d9e0b1147 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:14.164071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f45a5-7891-4d4b-92a1-bf0d9e0b1147 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:14.164309 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8171b77-bceb-4451-8a2f-95d354478c82 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:14.170989 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f45a5-7891-4d4b-92a1-bf0d9e0b1147 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:14.172450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f45a5-7891-4d4b-92a1-bf0d9e0b1147 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:14.176470 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:14.185961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-bd7f45a5-7891-4d4b-92a1-bf0d9e0b1147 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:14.185961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:14.185961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:14.185961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:14.185961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:14.185961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:14.185961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:14.185961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:14.185961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:14.185961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:14.185961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:14.185961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:14.185961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:14.185961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:14.185961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:14.186889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:14.186889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:14.186889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:14.186889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:14.186889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:14.186889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:14.186889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:14.186889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:14.186889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:14.186889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:14.186889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:14.186889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:14.186889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:14.186889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:14.186889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:14.186889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:14.186889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:14.187817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:14.187817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:14.187817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:14.187817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:14.187817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:14.187817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:14.187817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:14.187817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:14.187817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 324/659] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:14.188319 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f45a5-7891-4d4b-92a1-bf0d9e0b1147 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:14.236313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:14.263498 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8171b77-bceb-4451-8a2f-95d354478c82 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:14.284630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f45a5-7891-4d4b-92a1-bf0d9e0b1147 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:14.297446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9430b-69bd-4e94-872b-730734e0c045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:14.298253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9430b-69bd-4e94-872b-730734e0c045 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:14.298538 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9430b-69bd-4e94-872b-730734e0c045 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:14.298815 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f9430b-69bd-4e94-872b-730734e0c045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:14.309096 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9430b-69bd-4e94-872b-730734e0c045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:14.310556 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f9430b-69bd-4e94-872b-730734e0c045 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:14.317740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ce744-1da5-46ac-8cc9-8316e5b5564e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:14.318532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ce744-1da5-46ac-8cc9-8316e5b5564e None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:14.318813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ce744-1da5-46ac-8cc9-8316e5b5564e None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:14.319113 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ce744-1da5-46ac-8cc9-8316e5b5564e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:14.327366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-25f9430b-69bd-4e94-872b-730734e0c045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:14.327366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:14.327366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:14.327366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:14.327366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:14.327366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:14.327366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:14.327366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:14.327366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:14.327366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:14.327366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:14.327366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:14.327366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:14.327366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:14.327366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:14.328408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:14.328408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:14.328408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:14.328408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:14.328408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:14.328408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:14.328408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:14.328408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:14.328408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:14.328408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:14.328408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:14.328408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:14.328408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:14.328408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:14.328408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:14.328408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:14.328408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:14.329496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:14.329496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:14.329496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:14.329496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:14.329496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 336/660] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:14.329826 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9430b-69bd-4e94-872b-730734e0c045 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:14.330882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ce744-1da5-46ac-8cc9-8316e5b5564e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:14.339537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-211ce744-1da5-46ac-8cc9-8316e5b5564e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:14.339537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:14.339537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:14.339537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:14.339537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:14.339537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:14.339537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:14.339537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:14.339537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:14.339537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:14.339537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:14.339537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:14.339537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:14.339537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:14.339537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:14.339537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:14.340498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:14.340498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:14.340498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:14.340498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:14.340498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:14.340498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:14.340498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:14.340498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:14.340498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:14.340498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:14.340498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:14.340498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:14.340498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:14.340498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:14.340498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:14.340498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:14.341667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:14.341667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:14.341667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:14.341667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:14.341667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:14.341667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:14.341667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:14.341667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:14.341667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:14.341667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:14.341667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:14.341667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:14.341667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 325/661] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:31:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:14.341667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ce744-1da5-46ac-8cc9-8316e5b5564e None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:14.392320 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:14.434060 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f9430b-69bd-4e94-872b-730734e0c045 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:14.438989 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ce744-1da5-46ac-8cc9-8316e5b5564e None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:14.448904 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:14.466044 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8536273-b448-4a82-9c1a-79629e7cf4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:14.466702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8536273-b448-4a82-9c1a-79629e7cf4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:14.466982 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8536273-b448-4a82-9c1a-79629e7cf4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:14.467297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8536273-b448-4a82-9c1a-79629e7cf4b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:14.473500 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4e0bb-bcaf-4a92-b87b-57f321a64c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:14.474150 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4e0bb-bcaf-4a92-b87b-57f321a64c2a None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:14.474431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4e0bb-bcaf-4a92-b87b-57f321a64c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:14.474704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4e0bb-bcaf-4a92-b87b-57f321a64c2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:14.475580 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4e0bb-bcaf-4a92-b87b-57f321a64c2a None admin] RBAC: Adding query filter params (role.id=bb94781ab8054924a5cb21b561b12bbb, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=928d014648cd41d7b58e7e01845eb748) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:14.475888 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4e0bb-bcaf-4a92-b87b-57f321a64c2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:14.477398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4e0bb-bcaf-4a92-b87b-57f321a64c2a None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:14.483185 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 326/662] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:14 2022] GET /identity/v3/role_assignments?user.id=928d014648cd41d7b58e7e01845eb748&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=bb94781ab8054924a5cb21b561b12bbb => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:14.483951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4e0bb-bcaf-4a92-b87b-57f321a64c2a None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:14.558482 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8536273-b448-4a82-9c1a-79629e7cf4b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:14.560048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8536273-b448-4a82-9c1a-79629e7cf4b1 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:14.593649 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4e0bb-bcaf-4a92-b87b-57f321a64c2a None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:14.627588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:14.629173 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f456b-1aae-4ffa-9e5f-40fc4c212e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:14.629790 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f456b-1aae-4ffa-9e5f-40fc4c212e8e None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:14.630069 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f456b-1aae-4ffa-9e5f-40fc4c212e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:14.630341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f456b-1aae-4ffa-9e5f-40fc4c212e8e None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:14.631171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f456b-1aae-4ffa-9e5f-40fc4c212e8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:14.631484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f456b-1aae-4ffa-9e5f-40fc4c212e8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:14.633019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f456b-1aae-4ffa-9e5f-40fc4c212e8e None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:14.650756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 327/663] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:14.650756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f456b-1aae-4ffa-9e5f-40fc4c212e8e None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:14.689312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 337/664] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:31:14.690046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8536273-b448-4a82-9c1a-79629e7cf4b1 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:14.697209 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:14.767916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f456b-1aae-4ffa-9e5f-40fc4c212e8e None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:14.799239 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8536273-b448-4a82-9c1a-79629e7cf4b1 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:14.802304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab40e48-b7f8-49c4-b135-4893b93b0f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:14.803044 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab40e48-b7f8-49c4-b135-4893b93b0f33 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:14.803448 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab40e48-b7f8-49c4-b135-4893b93b0f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:14.803838 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab40e48-b7f8-49c4-b135-4893b93b0f33 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:14.804728 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab40e48-b7f8-49c4-b135-4893b93b0f33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:14.805123 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab40e48-b7f8-49c4-b135-4893b93b0f33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:14.806592 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab40e48-b7f8-49c4-b135-4893b93b0f33 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:14.818436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 328/665] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:14.819491 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab40e48-b7f8-49c4-b135-4893b93b0f33 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:14.836842 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279370db-2a3d-4aa3-8c90-0a9cf822e54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:14.837516 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279370db-2a3d-4aa3-8c90-0a9cf822e54f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:14.837795 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279370db-2a3d-4aa3-8c90-0a9cf822e54f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:14.838064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279370db-2a3d-4aa3-8c90-0a9cf822e54f None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:14.838802 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279370db-2a3d-4aa3-8c90-0a9cf822e54f None admin] RBAC: Adding query filter params (name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:14.839132 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279370db-2a3d-4aa3-8c90-0a9cf822e54f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:14.840507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279370db-2a3d-4aa3-8c90-0a9cf822e54f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:14.853152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 338/666] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:31:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:14.853152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279370db-2a3d-4aa3-8c90-0a9cf822e54f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:14.876084 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:14.925074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:14.932524 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab40e48-b7f8-49c4-b135-4893b93b0f33 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:14.960257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279370db-2a3d-4aa3-8c90-0a9cf822e54f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:14.967458 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e432f-0d71-4b5d-a0db-fa1f2d84eacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:14.968117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e432f-0d71-4b5d-a0db-fa1f2d84eacd None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:14.968400 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e432f-0d71-4b5d-a0db-fa1f2d84eacd None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:14.968676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e432f-0d71-4b5d-a0db-fa1f2d84eacd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:14.969508 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e432f-0d71-4b5d-a0db-fa1f2d84eacd None admin] RBAC: Adding query filter params (role.id=bb94781ab8054924a5cb21b561b12bbb, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=2707c6967a0a419a98ae142ff1d1a961) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:14.969821 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e432f-0d71-4b5d-a0db-fa1f2d84eacd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:14.971297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e432f-0d71-4b5d-a0db-fa1f2d84eacd None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:14.979059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 329/667] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:14 2022] GET /identity/v3/role_assignments?user.id=2707c6967a0a419a98ae142ff1d1a961&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=bb94781ab8054924a5cb21b561b12bbb => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:14.979059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e432f-0d71-4b5d-a0db-fa1f2d84eacd None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:15.007429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b8373-f0c3-4dea-8caf-c99459072140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:15.008273 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b8373-f0c3-4dea-8caf-c99459072140 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:15.008554 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b8373-f0c3-4dea-8caf-c99459072140 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:15.008823 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b8373-f0c3-4dea-8caf-c99459072140 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:15.021474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b8373-f0c3-4dea-8caf-c99459072140 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:15.022991 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b8373-f0c3-4dea-8caf-c99459072140 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:15.043541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-0c1b8373-f0c3-4dea-8caf-c99459072140 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:15.043541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:15.043541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:15.043541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:15.043541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:15.043541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:15.043541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:15.043541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:15.043541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:15.043541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:15.043541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:15.043541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:15.043541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:15.043541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:15.043541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:15.044531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:15.044531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:15.044531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:15.044531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:15.044531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:15.044531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:15.044531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:15.044531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:15.044531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:15.044531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:15.044531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:15.044531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:15.044531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:15.044531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:15.044531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:15.044531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:15.044531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:15.045431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:15.045431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:15.045431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:15.045431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:15.049522 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 339/668] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:15.050500 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b8373-f0c3-4dea-8caf-c99459072140 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:15.115308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e432f-0d71-4b5d-a0db-fa1f2d84eacd None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:15.152862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babfedd5-d75a-4ba1-9a1b-114122fc2c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:15.153676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfedd5-d75a-4ba1-9a1b-114122fc2c1e None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:15.153959 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfedd5-d75a-4ba1-9a1b-114122fc2c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:15.154231 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babfedd5-d75a-4ba1-9a1b-114122fc2c1e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:15.156128 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:15.164289 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babfedd5-d75a-4ba1-9a1b-114122fc2c1e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:15.166074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babfedd5-d75a-4ba1-9a1b-114122fc2c1e None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:15.184878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-babfedd5-d75a-4ba1-9a1b-114122fc2c1e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 05:31:15.184878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:15.184878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:15.184878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:15.184878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:15.184878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:15.184878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:15.184878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:15.184878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:15.184878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:15.184878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:15.184878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:15.184878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:15.184878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:15.184878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:15.184878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:15.185899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:15.186842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:15.186842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:15.186842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:15.186842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:15.186842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:15.186842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:15.186842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:15.186842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:15.186842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 21 05:31:15.186842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:15.189205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 330/669] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:31:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:15.190189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b8373-f0c3-4dea-8caf-c99459072140 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:15.190384 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babfedd5-d75a-4ba1-9a1b-114122fc2c1e None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:15.213184 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:15.236139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00501069-72a0-49ed-8c64-b29ef99e7318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:15.236952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00501069-72a0-49ed-8c64-b29ef99e7318 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:15.237232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00501069-72a0-49ed-8c64-b29ef99e7318 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:15.237508 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00501069-72a0-49ed-8c64-b29ef99e7318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:15.250616 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00501069-72a0-49ed-8c64-b29ef99e7318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:15.252495 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00501069-72a0-49ed-8c64-b29ef99e7318 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:15.273626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-00501069-72a0-49ed-8c64-b29ef99e7318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:15.273626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:15.273626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:15.273626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:15.273626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:15.273626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:15.273626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:15.273626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:15.273626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:15.273626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:15.273626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:15.273626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:15.273626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:15.273626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:15.273626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:15.274596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:15.274596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:15.274596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:15.274596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:15.274596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:15.274596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:15.274596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:15.274596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:15.274596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:15.274596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:15.274596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:15.274596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:15.274596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:15.274596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:15.274596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:15.274596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:15.274596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:15.275563 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:15.275563 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:15.275563 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:15.275563 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:15.275563 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 340/670] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:15.276428 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00501069-72a0-49ed-8c64-b29ef99e7318 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:15.326642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babfedd5-d75a-4ba1-9a1b-114122fc2c1e None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:15.371347 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7809b-8caa-4349-8737-12b61af46a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:15.372289 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7809b-8caa-4349-8737-12b61af46a48 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:15.372574 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7809b-8caa-4349-8737-12b61af46a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:15.372850 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7809b-8caa-4349-8737-12b61af46a48 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:15.373588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7809b-8caa-4349-8737-12b61af46a48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:15.373896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7809b-8caa-4349-8737-12b61af46a48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:15.376019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7809b-8caa-4349-8737-12b61af46a48 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:15.391287 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 331/671] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:15.396664 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7809b-8caa-4349-8737-12b61af46a48 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:15.401944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00501069-72a0-49ed-8c64-b29ef99e7318 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:15.435908 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:15.450986 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0256b83-54de-4fbd-b851-483fef6c582d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:15.451963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0256b83-54de-4fbd-b851-483fef6c582d None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:15.452354 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0256b83-54de-4fbd-b851-483fef6c582d None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:15.452710 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0256b83-54de-4fbd-b851-483fef6c582d None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:15.453942 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0256b83-54de-4fbd-b851-483fef6c582d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:15.455004 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0256b83-54de-4fbd-b851-483fef6c582d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:15.456625 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0256b83-54de-4fbd-b851-483fef6c582d None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:15.489509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 341/672] 10.222.0.44 () {56 vars in 1139 bytes} [Thu Jul 21 05:31:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:15.490631 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0256b83-54de-4fbd-b851-483fef6c582d None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:15.551551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7809b-8caa-4349-8737-12b61af46a48 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:15.595387 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:15.603826 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe674f-b02f-4993-8506-5cfe434f4f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:15.604512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe674f-b02f-4993-8506-5cfe434f4f3b None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:15.604795 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe674f-b02f-4993-8506-5cfe434f4f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:15.605069 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe674f-b02f-4993-8506-5cfe434f4f3b None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:15.605810 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe674f-b02f-4993-8506-5cfe434f4f3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:15.606118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe674f-b02f-4993-8506-5cfe434f4f3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:15.607544 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe674f-b02f-4993-8506-5cfe434f4f3b None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:15.612474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0256b83-54de-4fbd-b851-483fef6c582d None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:15.624014 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 332/673] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:15.624919 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe674f-b02f-4993-8506-5cfe434f4f3b None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:15.651141 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440824c-97e0-489c-9de7-6c0d933de16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:15.652309 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440824c-97e0-489c-9de7-6c0d933de16a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:15.652787 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440824c-97e0-489c-9de7-6c0d933de16a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:15.653251 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440824c-97e0-489c-9de7-6c0d933de16a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:15.661142 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440824c-97e0-489c-9de7-6c0d933de16a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:15.663665 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440824c-97e0-489c-9de7-6c0d933de16a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:15.678674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-a440824c-97e0-489c-9de7-6c0d933de16a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:15.678674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:15.678674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:15.678674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:15.678674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:15.678674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:15.678674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:15.678674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:15.678674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:15.678674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:15.678674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:15.678674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:15.678674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:15.678674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:15.678674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:15.679871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:15.679871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:15.679871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:15.679871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:15.679871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:15.679871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:15.679871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:15.679871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:15.679871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:15.679871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:15.679871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:15.679871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:15.679871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:15.679871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:15.679871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:15.679871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:15.679871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:15.680958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:15.680958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:15.680958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:15.680958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:15.680958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:15.680958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:15.680958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:15.680958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:15.683061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 342/674] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:15.683991 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440824c-97e0-489c-9de7-6c0d933de16a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:15.734213 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:15.748476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe674f-b02f-4993-8506-5cfe434f4f3b None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:15.783525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55673261-e1f5-43f6-9408-e2a9e03f0173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:15.784195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55673261-e1f5-43f6-9408-e2a9e03f0173 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:15.784482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55673261-e1f5-43f6-9408-e2a9e03f0173 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:15.784759 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55673261-e1f5-43f6-9408-e2a9e03f0173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:15.794614 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440824c-97e0-489c-9de7-6c0d933de16a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:15.795163 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:15.832149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e1951-3283-43d0-9577-26f45984ff76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:15.833212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1951-3283-43d0-9577-26f45984ff76 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:15.833665 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1951-3283-43d0-9577-26f45984ff76 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:15.834075 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e1951-3283-43d0-9577-26f45984ff76 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:15.842384 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e1951-3283-43d0-9577-26f45984ff76 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:15.844275 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e1951-3283-43d0-9577-26f45984ff76 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:15.859102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-f80e1951-3283-43d0-9577-26f45984ff76 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 05:31:15.859102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:15.859102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:15.859102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:15.859102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:15.859102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:15.859102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:15.859102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:15.859102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:15.859102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:15.859102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:15.859102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:15.859102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:15.859102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:15.859102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:15.859102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:15.860266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:15.861313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:15.861313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:15.861313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:15.861313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:15.861313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:15.861313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:15.861313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:15.861313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:15.861313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 05:31:15.861313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:15.863179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 343/675] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:31:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:15.864155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e1951-3283-43d0-9577-26f45984ff76 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:15.872978 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55673261-e1f5-43f6-9408-e2a9e03f0173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:15.874914 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55673261-e1f5-43f6-9408-e2a9e03f0173 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:15.974071 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e1951-3283-43d0-9577-26f45984ff76 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:15.977047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:15.998557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 333/676] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:31:15.999491 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55673261-e1f5-43f6-9408-e2a9e03f0173 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:16.015685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8aa5f-5ba7-4f19-8d05-3add9f18bab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:16.017048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8aa5f-5ba7-4f19-8d05-3add9f18bab5 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:16.017547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8aa5f-5ba7-4f19-8d05-3add9f18bab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:16.018068 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8aa5f-5ba7-4f19-8d05-3add9f18bab5 None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:16.019211 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8aa5f-5ba7-4f19-8d05-3add9f18bab5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:16.019824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8aa5f-5ba7-4f19-8d05-3add9f18bab5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:16.021484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8aa5f-5ba7-4f19-8d05-3add9f18bab5 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:16.035056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 344/677] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:16.036151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8aa5f-5ba7-4f19-8d05-3add9f18bab5 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:16.044764 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:16.106652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55673261-e1f5-43f6-9408-e2a9e03f0173 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:16.142403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8aa5f-5ba7-4f19-8d05-3add9f18bab5 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:16.143475 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ea40d-5961-4ece-a3ef-d389fc3a12a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:16.144154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ea40d-5961-4ece-a3ef-d389fc3a12a4 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:16.144479 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ea40d-5961-4ece-a3ef-d389fc3a12a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:16.144755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ea40d-5961-4ece-a3ef-d389fc3a12a4 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:16.145767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ea40d-5961-4ece-a3ef-d389fc3a12a4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:16.146073 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ea40d-5961-4ece-a3ef-d389fc3a12a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:16.148346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ea40d-5961-4ece-a3ef-d389fc3a12a4 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:16.171357 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 334/678] 10.222.0.44 () {56 vars in 1135 bytes} [Thu Jul 21 05:31:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:16.172798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ea40d-5961-4ece-a3ef-d389fc3a12a4 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:16.183125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df3a1d-6dfa-4257-a925-503c16f9922a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:16.183640 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df3a1d-6dfa-4257-a925-503c16f9922a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:16.183922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df3a1d-6dfa-4257-a925-503c16f9922a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:16.184196 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df3a1d-6dfa-4257-a925-503c16f9922a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:16.196081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df3a1d-6dfa-4257-a925-503c16f9922a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:16.198133 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df3a1d-6dfa-4257-a925-503c16f9922a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:16.222248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-19df3a1d-6dfa-4257-a925-503c16f9922a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:16.222248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:16.222248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:16.222248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:16.222248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:16.222248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:16.222248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:16.222248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:16.222248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:16.222248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:16.222248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:16.222248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:16.222248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:16.222248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:16.222248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:16.223237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:16.223237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:16.223237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:16.223237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:16.223237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:16.223237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:16.223237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:16.223237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:16.223237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:16.223237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:16.223237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:16.223237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:16.223237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:16.223237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:16.223237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:16.223237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:16.223237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:16.224197 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:16.224197 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:16.224197 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:16.224197 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:16.229917 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 345/679] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:16.229917 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df3a1d-6dfa-4257-a925-503c16f9922a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:16.250349 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:16.290218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ea40d-5961-4ece-a3ef-d389fc3a12a4 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:16.319589 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:16.326825 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28fe5b-641f-4203-ac6d-1ec9263298a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:16.327606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28fe5b-641f-4203-ac6d-1ec9263298a9 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:16.327891 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28fe5b-641f-4203-ac6d-1ec9263298a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:16.328168 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28fe5b-641f-4203-ac6d-1ec9263298a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:16.329099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28fe5b-641f-4203-ac6d-1ec9263298a9 None admin] RBAC: Adding query filter params (role.id=bb94781ab8054924a5cb21b561b12bbb, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=f3f76371c6a04fc5a477d6ef1908a08e) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:16.329611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28fe5b-641f-4203-ac6d-1ec9263298a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:16.330929 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a28fe5b-641f-4203-ac6d-1ec9263298a9 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:16.343324 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 335/680] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:16 2022] GET /identity/v3/role_assignments?user.id=f3f76371c6a04fc5a477d6ef1908a08e&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=bb94781ab8054924a5cb21b561b12bbb => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:16.344388 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28fe5b-641f-4203-ac6d-1ec9263298a9 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:16.353912 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df3a1d-6dfa-4257-a925-503c16f9922a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:16.393986 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd80cb7-67be-417c-8458-452aacd0a9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:16.395526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd80cb7-67be-417c-8458-452aacd0a9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:16.396016 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd80cb7-67be-417c-8458-452aacd0a9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:16.396509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd80cb7-67be-417c-8458-452aacd0a9f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:16.478979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd80cb7-67be-417c-8458-452aacd0a9f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:16.481531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd80cb7-67be-417c-8458-452aacd0a9f9 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:16.502949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:16.555953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a28fe5b-641f-4203-ac6d-1ec9263298a9 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:16.602826 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 346/681] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:31:16.608620 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd80cb7-67be-417c-8458-452aacd0a9f9 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:16.628690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f201fa0-b949-4c45-81ba-49a73e0ea0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:16.629860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f201fa0-b949-4c45-81ba-49a73e0ea0ce None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:16.630397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f201fa0-b949-4c45-81ba-49a73e0ea0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:16.631277 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f201fa0-b949-4c45-81ba-49a73e0ea0ce None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:16.632303 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f201fa0-b949-4c45-81ba-49a73e0ea0ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:16.633149 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f201fa0-b949-4c45-81ba-49a73e0ea0ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:16.634727 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f201fa0-b949-4c45-81ba-49a73e0ea0ce None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:16.652472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 336/682] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:16.659506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f201fa0-b949-4c45-81ba-49a73e0ea0ce None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:16.689343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:16.728851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd80cb7-67be-417c-8458-452aacd0a9f9 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:16.746383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:16.766235 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a491f-0a11-46c3-9fba-7d37e74b6e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:16.767239 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a491f-0a11-46c3-9fba-7d37e74b6e39 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:16.767612 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a491f-0a11-46c3-9fba-7d37e74b6e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:16.767978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a491f-0a11-46c3-9fba-7d37e74b6e39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:16.780641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a491f-0a11-46c3-9fba-7d37e74b6e39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:16.782239 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a491f-0a11-46c3-9fba-7d37e74b6e39 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:16.789225 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f201fa0-b949-4c45-81ba-49a73e0ea0ce None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:16.800946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-4f9a491f-0a11-46c3-9fba-7d37e74b6e39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:16.800946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:16.800946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:16.800946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:16.800946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:16.800946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:16.800946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:16.800946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:16.800946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:16.800946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:16.800946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:16.800946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:16.800946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:16.800946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:16.800946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:16.801950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:16.801950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:16.801950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:16.801950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:16.801950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:16.801950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:16.801950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:16.801950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:16.801950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:16.801950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:16.801950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:16.801950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:16.801950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:16.801950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:16.801950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:16.801950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:16.801950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:16.802849 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:16.802849 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:16.802849 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:16.802849 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:16.802849 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 347/683] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:16.807052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780bee3-7ed1-42a7-af40-37bcd62f9136 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:16.807052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780bee3-7ed1-42a7-af40-37bcd62f9136 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:16.807052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780bee3-7ed1-42a7-af40-37bcd62f9136 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:16.807052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 348/684] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:16.811779 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780bee3-7ed1-42a7-af40-37bcd62f9136 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:16.833213 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14325eb-a8f6-4051-a5a2-350d397e18b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:16.834297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14325eb-a8f6-4051-a5a2-350d397e18b6 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:16.834672 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14325eb-a8f6-4051-a5a2-350d397e18b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:16.835058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14325eb-a8f6-4051-a5a2-350d397e18b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:16.842853 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14325eb-a8f6-4051-a5a2-350d397e18b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:16.844553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14325eb-a8f6-4051-a5a2-350d397e18b6 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:16.859412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-b14325eb-a8f6-4051-a5a2-350d397e18b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:16.859412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:16.859412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:16.859412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:16.859412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:16.859412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:16.859412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:16.859412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:16.859412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:16.859412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:16.859412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:16.859412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:16.859412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:16.859412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:16.859412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:16.860531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:16.860531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:16.860531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:16.860531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:16.860531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:16.860531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:16.860531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:16.860531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:16.860531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:16.860531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:16.860531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:16.860531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:16.860531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:16.860531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:16.860531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:16.860531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:16.860531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:16.861644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:16.861644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:16.861644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:16.861644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:16.861644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:16.861644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:16.861644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:16.861644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:16.866994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 337/685] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:16.866994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749fb63-bd46-4ee8-9ea5-6dfdaebfdb11 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:16.866994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749fb63-bd46-4ee8-9ea5-6dfdaebfdb11 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:16.866994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749fb63-bd46-4ee8-9ea5-6dfdaebfdb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:16.882788 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-0749fb63-bd46-4ee8-9ea5-6dfdaebfdb11 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:16.884367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-0749fb63-bd46-4ee8-9ea5-6dfdaebfdb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:16.916851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780bee3-7ed1-42a7-af40-37bcd62f9136 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:16.955919 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e6aba-72ab-4344-bf4f-39d5ba80c279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:16.956891 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e6aba-72ab-4344-bf4f-39d5ba80c279 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:16.957315 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e6aba-72ab-4344-bf4f-39d5ba80c279 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:16.957773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e6aba-72ab-4344-bf4f-39d5ba80c279 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:16.958627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e6aba-72ab-4344-bf4f-39d5ba80c279 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:16.959131 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e6aba-72ab-4344-bf4f-39d5ba80c279 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:16.960724 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e6aba-72ab-4344-bf4f-39d5ba80c279 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:16.961456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 338/686] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:31:16.962326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749fb63-bd46-4ee8-9ea5-6dfdaebfdb11 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:16.973937 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 349/687] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:16.975464 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:16.979782 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e6aba-72ab-4344-bf4f-39d5ba80c279 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:17.021675 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:17.067517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749fb63-bd46-4ee8-9ea5-6dfdaebfdb11 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:17.094883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e6aba-72ab-4344-bf4f-39d5ba80c279 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:17.109647 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850fb82-3297-48da-a620-17000d4a25f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:17.110292 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850fb82-3297-48da-a620-17000d4a25f0 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:17.110574 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850fb82-3297-48da-a620-17000d4a25f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:17.110847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850fb82-3297-48da-a620-17000d4a25f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:17.112257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850fb82-3297-48da-a620-17000d4a25f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:17.112706 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850fb82-3297-48da-a620-17000d4a25f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:17.114154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850fb82-3297-48da-a620-17000d4a25f0 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:17.128748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 339/688] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:17.129596 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850fb82-3297-48da-a620-17000d4a25f0 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:17.145157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd30ba3-bc01-472e-97c0-c71bd38cfa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:17.146081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd30ba3-bc01-472e-97c0-c71bd38cfa42 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:17.146361 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd30ba3-bc01-472e-97c0-c71bd38cfa42 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:17.146631 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd30ba3-bc01-472e-97c0-c71bd38cfa42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:17.153963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd30ba3-bc01-472e-97c0-c71bd38cfa42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:17.155540 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd30ba3-bc01-472e-97c0-c71bd38cfa42 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:17.170465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-afd30ba3-bc01-472e-97c0-c71bd38cfa42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:17.170465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:17.170465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:17.170465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:17.170465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:17.170465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:17.170465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:17.170465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:17.170465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:17.170465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:17.170465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:17.170465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:17.170465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:17.170465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:17.170465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:17.171446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:17.171446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:17.171446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:17.171446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:17.171446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:17.171446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:17.171446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:17.171446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:17.171446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:17.171446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:17.171446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:17.171446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:17.171446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:17.171446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:17.171446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:17.171446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:17.171446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:17.172356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:17.172356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:17.172356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:17.172356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:17.172356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:17.172356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:17.172356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:17.172356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:17.176547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 350/689] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:17.177394 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd30ba3-bc01-472e-97c0-c71bd38cfa42 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:17.202120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:17.235243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850fb82-3297-48da-a620-17000d4a25f0 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:17.265662 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:17.271828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b2e3d-81d9-422e-9483-d3e00de721b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:17.272578 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b2e3d-81d9-422e-9483-d3e00de721b8 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:17.272992 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b2e3d-81d9-422e-9483-d3e00de721b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:17.273209 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b2e3d-81d9-422e-9483-d3e00de721b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:17.280183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd30ba3-bc01-472e-97c0-c71bd38cfa42 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:17.283903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b2e3d-81d9-422e-9483-d3e00de721b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:17.291939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-bc4b2e3d-81d9-422e-9483-d3e00de721b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:31:17.291939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:17.291939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:17.291939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:17.291939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:17.291939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:17.291939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:17.291939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:17.291939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:17.291939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:17.291939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:17.291939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:17.291939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:17.291939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:17.291939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:17.291939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:17.292883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:17.292883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:17.292883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:17.292883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:17.292883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:17.292883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:17.292883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:17.292883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:17.292883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:17.292883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:17.292883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:17.292883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:17.292883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:17.292883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:17.292883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:17.292883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:17.293723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:17.293723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:17.293723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:17.293723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:17.293723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:17.293723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:17.293723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:17.293723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:17.293723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:17.293723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:17.293723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:31:17.293723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:17.293723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 340/690] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:31:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:17.294442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b2e3d-81d9-422e-9483-d3e00de721b8 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:17.322306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a11d6cd-ed11-4e62-ae8d-7647fbf6a6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:17.322972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11d6cd-ed11-4e62-ae8d-7647fbf6a6db None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:17.323280 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11d6cd-ed11-4e62-ae8d-7647fbf6a6db None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:17.323554 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11d6cd-ed11-4e62-ae8d-7647fbf6a6db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:17.324372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11d6cd-ed11-4e62-ae8d-7647fbf6a6db None admin] RBAC: Adding query filter params (role.id=bb94781ab8054924a5cb21b561b12bbb, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=e07508a9fa9146528620b7f8abec8549) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:17.324676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11d6cd-ed11-4e62-ae8d-7647fbf6a6db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:17.326212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a11d6cd-ed11-4e62-ae8d-7647fbf6a6db None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:17.339067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 351/691] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:17 2022] GET /identity/v3/role_assignments?user.id=e07508a9fa9146528620b7f8abec8549&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=bb94781ab8054924a5cb21b561b12bbb => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:17.339067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302da75-aba7-4523-a8f9-62ab3fe92f53 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:17.339067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302da75-aba7-4523-a8f9-62ab3fe92f53 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:17.339067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302da75-aba7-4523-a8f9-62ab3fe92f53 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:17.339067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 352/692] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:17.339067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302da75-aba7-4523-a8f9-62ab3fe92f53 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:17.414712 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b2e3d-81d9-422e-9483-d3e00de721b8 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:17.447645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:17.453747 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e5464-71fa-4c6b-85dd-a9091f7f68f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:17.454597 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e5464-71fa-4c6b-85dd-a9091f7f68f5 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:17.455417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e5464-71fa-4c6b-85dd-a9091f7f68f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:17.455714 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e5464-71fa-4c6b-85dd-a9091f7f68f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:17.457060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490e5464-71fa-4c6b-85dd-a9091f7f68f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:17.457188 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490e5464-71fa-4c6b-85dd-a9091f7f68f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:17.458813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490e5464-71fa-4c6b-85dd-a9091f7f68f5 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:17.476337 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302da75-aba7-4523-a8f9-62ab3fe92f53 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:17.476609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 341/693] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:17.477974 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6c8b3-84df-45ad-8036-e21a42145dfa None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:17.478260 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6c8b3-84df-45ad-8036-e21a42145dfa None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:17.478580 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6c8b3-84df-45ad-8036-e21a42145dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:17.500278 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-00e6c8b3-84df-45ad-8036-e21a42145dfa None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:17.501690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-00e6c8b3-84df-45ad-8036-e21a42145dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:17.519492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e320ce3-ca55-4039-b7d8-873d3d6fd2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:17.520526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e320ce3-ca55-4039-b7d8-873d3d6fd2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:17.521016 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e320ce3-ca55-4039-b7d8-873d3d6fd2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:17.521524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e320ce3-ca55-4039-b7d8-873d3d6fd2e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:17.522606 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e320ce3-ca55-4039-b7d8-873d3d6fd2e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:17.522883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e320ce3-ca55-4039-b7d8-873d3d6fd2e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:17.524342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e320ce3-ca55-4039-b7d8-873d3d6fd2e8 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:17.537785 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 353/694] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:31:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:17.538793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e320ce3-ca55-4039-b7d8-873d3d6fd2e8 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:17.561507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:17.607089 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 342/695] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:31:17.607089 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6c8b3-84df-45ad-8036-e21a42145dfa None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:17.692075 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e320ce3-ca55-4039-b7d8-873d3d6fd2e8 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:17.726063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a54d03-35d5-45b3-97ad-b4e55c1d4009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:17.727096 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6c8b3-84df-45ad-8036-e21a42145dfa None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:17.727447 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a54d03-35d5-45b3-97ad-b4e55c1d4009 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:17.728047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a54d03-35d5-45b3-97ad-b4e55c1d4009 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:17.728613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a54d03-35d5-45b3-97ad-b4e55c1d4009 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:17.729718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a54d03-35d5-45b3-97ad-b4e55c1d4009 None admin] RBAC: Adding query filter params (role.id=bb94781ab8054924a5cb21b561b12bbb, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=61b61188b5b94a51ba1ae1a88b78b89f) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:17.730260 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a54d03-35d5-45b3-97ad-b4e55c1d4009 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:17.732190 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a54d03-35d5-45b3-97ad-b4e55c1d4009 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:17.740142 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 354/696] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:17 2022] GET /identity/v3/role_assignments?user.id=61b61188b5b94a51ba1ae1a88b78b89f&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=bb94781ab8054924a5cb21b561b12bbb => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:17.741447 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a54d03-35d5-45b3-97ad-b4e55c1d4009 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:17.753272 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:17.774493 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6eb74a-2c06-4db2-ae06-14087ef9ace8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:17.775771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6eb74a-2c06-4db2-ae06-14087ef9ace8 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:17.777047 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6eb74a-2c06-4db2-ae06-14087ef9ace8 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:17.777580 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6eb74a-2c06-4db2-ae06-14087ef9ace8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:17.791897 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6eb74a-2c06-4db2-ae06-14087ef9ace8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:17.794369 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6eb74a-2c06-4db2-ae06-14087ef9ace8 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:17.817093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-1a6eb74a-2c06-4db2-ae06-14087ef9ace8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:17.817093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:17.817093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:17.817093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:17.817093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:17.817093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:17.817093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:17.817093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:17.817093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:17.817093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:17.817093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:17.817093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:17.817093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:17.817093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:17.817093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:17.818103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:17.818103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:17.818103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:17.818103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:17.818103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:17.818103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:17.818103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:17.818103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:17.818103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:17.818103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:17.818103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:17.818103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:17.818103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:17.818103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:17.818103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:17.818103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:17.818103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:17.818984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:17.818984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:17.818984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:17.818984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:17.821993 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 343/697] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:17.828253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6eb74a-2c06-4db2-ae06-14087ef9ace8 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:17.926926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:17.958342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a54d03-35d5-45b3-97ad-b4e55c1d4009 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:17.959626 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6eb74a-2c06-4db2-ae06-14087ef9ace8 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:17.997076 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41efcd-d74d-459d-87b4-ae6ce1aaf588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:17.997974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41efcd-d74d-459d-87b4-ae6ce1aaf588 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:17.998257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41efcd-d74d-459d-87b4-ae6ce1aaf588 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:17.998528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41efcd-d74d-459d-87b4-ae6ce1aaf588 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:18.004699 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93febd-111f-4070-b085-461b55c3eca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:18.005638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93febd-111f-4070-b085-461b55c3eca9 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:18.006621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93febd-111f-4070-b085-461b55c3eca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:18.007158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93febd-111f-4070-b085-461b55c3eca9 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:18.008568 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93febd-111f-4070-b085-461b55c3eca9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:18.009079 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93febd-111f-4070-b085-461b55c3eca9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:18.010919 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b93febd-111f-4070-b085-461b55c3eca9 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:18.011060 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41efcd-d74d-459d-87b4-ae6ce1aaf588 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:18.012403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41efcd-d74d-459d-87b4-ae6ce1aaf588 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:18.025318 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 344/698] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:18.030601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-4b41efcd-d74d-459d-87b4-ae6ce1aaf588 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:18.030601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:18.030601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:18.030601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:18.030601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:18.030601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:18.030601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:18.030601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:18.030601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:18.030601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:18.030601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:18.030601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:18.030601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:18.030601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:18.030601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:18.031882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:18.031882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:18.031882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:18.031882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:18.031882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:18.031882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:18.031882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:18.031882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:18.031882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:18.031882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:18.031882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:18.031882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:18.031882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:18.031882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:18.031882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:18.031882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:18.031882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:18.032773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:18.032773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:18.032773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:18.032773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:18.032773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 355/699] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:18.034421 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93febd-111f-4070-b085-461b55c3eca9 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:18.038097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41efcd-d74d-459d-87b4-ae6ce1aaf588 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:18.080055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:18.157666 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93febd-111f-4070-b085-461b55c3eca9 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:18.174061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41efcd-d74d-459d-87b4-ae6ce1aaf588 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:18.198143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722779c-3aca-40d3-8d7f-ba7e1a913293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:18.199375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722779c-3aca-40d3-8d7f-ba7e1a913293 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:18.199880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722779c-3aca-40d3-8d7f-ba7e1a913293 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:18.200348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722779c-3aca-40d3-8d7f-ba7e1a913293 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:18.208719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:18.211829 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722779c-3aca-40d3-8d7f-ba7e1a913293 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:18.214289 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722779c-3aca-40d3-8d7f-ba7e1a913293 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:18.216026 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67496bc-8e1e-498b-b111-85b330419e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:18.216950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67496bc-8e1e-498b-b111-85b330419e04 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:18.217243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67496bc-8e1e-498b-b111-85b330419e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:18.217514 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67496bc-8e1e-498b-b111-85b330419e04 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:18.218839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67496bc-8e1e-498b-b111-85b330419e04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:18.219204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67496bc-8e1e-498b-b111-85b330419e04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:18.220593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67496bc-8e1e-498b-b111-85b330419e04 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:18.233424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-a722779c-3aca-40d3-8d7f-ba7e1a913293 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:31:18.233424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:18.233424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:18.233424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:18.233424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:18.233424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:18.233424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:18.233424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:18.233424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:18.233424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:18.233424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:18.233424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:18.233424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:18.233424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:18.233424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:18.233424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:18.234415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:18.235379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:18.235379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:18.235379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:18.235379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:18.235379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:18.235379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:18.235379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:18.235379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:18.235379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:31:18.235379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:18.235906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 356/700] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:18.236582 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 345/701] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jul 21 05:31:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:18.240800 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67496bc-8e1e-498b-b111-85b330419e04 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:18.243282 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722779c-3aca-40d3-8d7f-ba7e1a913293 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:18.372149 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:18.383563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722779c-3aca-40d3-8d7f-ba7e1a913293 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:18.421540 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb5481-2a4d-4d6e-be04-9b5aed7e9823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:18.423142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb5481-2a4d-4d6e-be04-9b5aed7e9823 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:18.423620 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb5481-2a4d-4d6e-be04-9b5aed7e9823 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:18.424087 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb5481-2a4d-4d6e-be04-9b5aed7e9823 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:18.425178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb5481-2a4d-4d6e-be04-9b5aed7e9823 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:18.425676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb5481-2a4d-4d6e-be04-9b5aed7e9823 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:18.427937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb5481-2a4d-4d6e-be04-9b5aed7e9823 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:18.451507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 346/702] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jul 21 05:31:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:18.453607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb5481-2a4d-4d6e-be04-9b5aed7e9823 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:18.544967 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67496bc-8e1e-498b-b111-85b330419e04 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:18.559001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb5481-2a4d-4d6e-be04-9b5aed7e9823 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:18.618281 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cb9b8-cd50-4e0e-a850-910f8affc737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:18.619375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cb9b8-cd50-4e0e-a850-910f8affc737 None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:18.619653 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578f7f3-e830-48f1-b1cd-f34e2f1c648d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:18.620507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578f7f3-e830-48f1-b1cd-f34e2f1c648d None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:18.621001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cb9b8-cd50-4e0e-a850-910f8affc737 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:18.621488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cb9b8-cd50-4e0e-a850-910f8affc737 None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:18.621948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578f7f3-e830-48f1-b1cd-f34e2f1c648d None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:18.622905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cb9b8-cd50-4e0e-a850-910f8affc737 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:18.623440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578f7f3-e830-48f1-b1cd-f34e2f1c648d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:18.624897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cb9b8-cd50-4e0e-a850-910f8affc737 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:18.659131 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 357/703] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:31:18 2022] POST /identity/v3/users => generated 254 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:31:18.675755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:18.729826 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578f7f3-e830-48f1-b1cd-f34e2f1c648d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:18.731356 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578f7f3-e830-48f1-b1cd-f34e2f1c648d None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:18.752013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d9dd0-94d6-4fdd-8f01-abde3d6a9a0b None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:18.752683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d9dd0-94d6-4fdd-8f01-abde3d6a9a0b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:18.753229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d9dd0-94d6-4fdd-8f01-abde3d6a9a0b None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:18.759634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 358/704] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:31:18.761385 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006b12b-233f-447d-ac51-49ed9c37d3fa None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:18.761868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006b12b-233f-447d-ac51-49ed9c37d3fa None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:18.762353 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006b12b-233f-447d-ac51-49ed9c37d3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:18.780335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-4006b12b-233f-447d-ac51-49ed9c37d3fa None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:18.781904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4006b12b-233f-447d-ac51-49ed9c37d3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:18.832772 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:18.858940 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 359/705] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 21 05:31:18.868437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 347/706] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 21 05:31:18.874310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006b12b-233f-447d-ac51-49ed9c37d3fa None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:18.938476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578f7f3-e830-48f1-b1cd-f34e2f1c648d None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:18.997985 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006b12b-233f-447d-ac51-49ed9c37d3fa None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:18.998559 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:19.049054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8f140-b3be-4a32-a37c-2e2706be9237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:19.049972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8f140-b3be-4a32-a37c-2e2706be9237 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:19.050258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8f140-b3be-4a32-a37c-2e2706be9237 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:19.050533 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8f140-b3be-4a32-a37c-2e2706be9237 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:19.062064 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578f7f3-e830-48f1-b1cd-f34e2f1c648d None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:19.069452 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8f140-b3be-4a32-a37c-2e2706be9237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:19.071007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8f140-b3be-4a32-a37c-2e2706be9237 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:19.091594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-9ae8f140-b3be-4a32-a37c-2e2706be9237 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:19.091594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:19.091594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:19.091594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:19.091594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:19.091594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:19.091594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:19.091594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:19.091594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:19.091594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:19.091594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:19.091594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:19.091594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:19.091594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:19.091594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:19.092582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:19.092582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:19.092582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:19.092582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:19.092582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:19.092582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:19.092582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:19.092582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:19.092582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:19.092582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:19.092582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:19.092582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:19.092582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:19.092582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:19.092582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:19.092582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:19.092582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:19.093497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:19.093497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:19.093497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:19.093497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:19.093718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 360/707] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:19.099858 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8f140-b3be-4a32-a37c-2e2706be9237 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:19.102133 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7238f-a526-4792-a086-89b0d031f49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:19.103214 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7238f-a526-4792-a086-89b0d031f49f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:19.103693 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7238f-a526-4792-a086-89b0d031f49f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:19.104159 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced7238f-a526-4792-a086-89b0d031f49f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:19.105028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7238f-a526-4792-a086-89b0d031f49f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:19.106604 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced7238f-a526-4792-a086-89b0d031f49f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:19.121873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-ced7238f-a526-4792-a086-89b0d031f49f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 05:31:19.121873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:19.121873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:19.121873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:19.121873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:19.121873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:19.121873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:19.121873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:19.121873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 05:31:19.121873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:19.121873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:19.121873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:19.121873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:19.121873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:19.121873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:19.121873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:19.122881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:19.122881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:19.122881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:19.122881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:19.122881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:19.122881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:19.122881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:19.122881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:19.122881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:19.122881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:19.122881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:19.122881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:19.122881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:19.122881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:19.122881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:19.122881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:19.123749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:19.123749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:19.123749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:19.123749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:19.123749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:19.123749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:19.123749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:19.123749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:19.123749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:19.123749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:19.123749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:19.123749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:19.123749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:19.123749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 05:31:19.123749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:19.126714 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 348/708] 10.222.0.44 () {56 vars in 1129 bytes} [Thu Jul 21 05:31:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:19.133678 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7238f-a526-4792-a086-89b0d031f49f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:19.147552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:19.245457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8f140-b3be-4a32-a37c-2e2706be9237 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:19.259852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced7238f-a526-4792-a086-89b0d031f49f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:19.278047 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:19.298613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48948807-2c15-4595-87e8-1b0915fc4981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:19.300865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48948807-2c15-4595-87e8-1b0915fc4981 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:19.302191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48948807-2c15-4595-87e8-1b0915fc4981 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:19.302900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48948807-2c15-4595-87e8-1b0915fc4981 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:19.305384 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48948807-2c15-4595-87e8-1b0915fc4981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:19.306787 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48948807-2c15-4595-87e8-1b0915fc4981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:19.312148 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48948807-2c15-4595-87e8-1b0915fc4981 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:19.315510 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca6467-db48-4046-a600-048112ae1ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:19.317177 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca6467-db48-4046-a600-048112ae1ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:19.317714 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca6467-db48-4046-a600-048112ae1ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:19.318674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca6467-db48-4046-a600-048112ae1ad6 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:19.319660 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca6467-db48-4046-a600-048112ae1ad6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:19.321054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca6467-db48-4046-a600-048112ae1ad6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:19.324360 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca6467-db48-4046-a600-048112ae1ad6 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:19.333971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 349/709] 10.222.0.44 () {56 vars in 1124 bytes} [Thu Jul 21 05:31:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:19.336478 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 361/710] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:19.344119 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca6467-db48-4046-a600-048112ae1ad6 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:19.344321 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48948807-2c15-4595-87e8-1b0915fc4981 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:19.462930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca6467-db48-4046-a600-048112ae1ad6 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:19.507307 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:19.531340 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48948807-2c15-4595-87e8-1b0915fc4981 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:19.532278 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397b875-3b59-4ccd-b7d7-6c4af32d0725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:19.534701 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397b875-3b59-4ccd-b7d7-6c4af32d0725 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:19.535187 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397b875-3b59-4ccd-b7d7-6c4af32d0725 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:19.535634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e397b875-3b59-4ccd-b7d7-6c4af32d0725 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:19.550996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397b875-3b59-4ccd-b7d7-6c4af32d0725 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:19.553992 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e397b875-3b59-4ccd-b7d7-6c4af32d0725 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:19.582210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-e397b875-3b59-4ccd-b7d7-6c4af32d0725 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:19.582210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:19.582210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:19.582210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:19.582210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:19.582210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:19.582210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:19.582210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:19.582210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:19.582210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:19.582210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:19.582210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:19.582210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:19.582210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:19.582210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:19.583224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:19.583224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:19.583224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:19.583224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:19.583224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:19.583224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:19.583224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:19.583224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:19.583224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:19.583224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:19.583224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:19.583224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:19.583224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:19.583224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:19.583224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:19.583224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:19.583224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:19.584143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:19.584143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:19.584143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:19.584143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:19.584143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:19.584143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:19.584143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:19.584143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:19.592009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 350/711] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:19.597227 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383f95e-9879-4843-bac5-e1ecaecd553f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:19.598125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383f95e-9879-4843-bac5-e1ecaecd553f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:19.598595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383f95e-9879-4843-bac5-e1ecaecd553f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:19.599160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383f95e-9879-4843-bac5-e1ecaecd553f None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:19.600009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383f95e-9879-4843-bac5-e1ecaecd553f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:19.600454 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383f95e-9879-4843-bac5-e1ecaecd553f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:19.602214 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383f95e-9879-4843-bac5-e1ecaecd553f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:19.603303 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397b875-3b59-4ccd-b7d7-6c4af32d0725 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:19.619372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 362/712] 10.222.0.44 () {56 vars in 1124 bytes} [Thu Jul 21 05:31:19 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:19.655760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:19.715746 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e397b875-3b59-4ccd-b7d7-6c4af32d0725 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:19.752619 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d92c5e-d476-4a7f-8464-de9b6c3e1a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:19.753299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d92c5e-d476-4a7f-8464-de9b6c3e1a8d None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:19.753599 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d92c5e-d476-4a7f-8464-de9b6c3e1a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:19.753847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d92c5e-d476-4a7f-8464-de9b6c3e1a8d None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:19.754629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d92c5e-d476-4a7f-8464-de9b6c3e1a8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:19.754932 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d92c5e-d476-4a7f-8464-de9b6c3e1a8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:19.756550 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d92c5e-d476-4a7f-8464-de9b6c3e1a8d None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:19.770869 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 351/713] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:19.777108 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383f95e-9879-4843-bac5-e1ecaecd553f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:19.887765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383f95e-9879-4843-bac5-e1ecaecd553f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:19.922696 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6ce9e-b31c-4b7d-8c6d-5b7e1ab09c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:19.925137 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6ce9e-b31c-4b7d-8c6d-5b7e1ab09c64 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:19.925416 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6ce9e-b31c-4b7d-8c6d-5b7e1ab09c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:19.925686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6ce9e-b31c-4b7d-8c6d-5b7e1ab09c64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:19.926523 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6ce9e-b31c-4b7d-8c6d-5b7e1ab09c64 None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=928d014648cd41d7b58e7e01845eb748) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:19.926828 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6ce9e-b31c-4b7d-8c6d-5b7e1ab09c64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:19.928295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6ce9e-b31c-4b7d-8c6d-5b7e1ab09c64 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:19.934543 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 363/714] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:19 2022] GET /identity/v3/role_assignments?user.id=928d014648cd41d7b58e7e01845eb748&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:20.010005 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ba211-148d-488b-abc3-93ce49185ec2 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:20.010316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ba211-148d-488b-abc3-93ce49185ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:20.010482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ba211-148d-488b-abc3-93ce49185ec2 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:20.012203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 352/715] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:20.017823 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c37669-0e70-43b5-b01d-9c648233b82a None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:20.018111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c37669-0e70-43b5-b01d-9c648233b82a None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:20.018386 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c37669-0e70-43b5-b01d-9c648233b82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:20.036564 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-74c37669-0e70-43b5-b01d-9c648233b82a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:20.038100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-74c37669-0e70-43b5-b01d-9c648233b82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:20.052248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:20.124376 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 364/716] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:31:20.148370 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ba211-148d-488b-abc3-93ce49185ec2 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:20.248058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:20.257864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ba211-148d-488b-abc3-93ce49185ec2 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:20.302636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d819e-b3e1-4b3d-aa92-70eca6f2983a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:20.303580 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d819e-b3e1-4b3d-aa92-70eca6f2983a None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:20.303890 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d819e-b3e1-4b3d-aa92-70eca6f2983a None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:20.304219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d819e-b3e1-4b3d-aa92-70eca6f2983a None admin] PATH_INFO: `/v3/services/network` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:20.304896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d819e-b3e1-4b3d-aa92-70eca6f2983a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:20.306394 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d819e-b3e1-4b3d-aa92-70eca6f2983a None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:20.325224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-008d819e-b3e1-4b3d-aa92-70eca6f2983a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 05:31:20.325224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:20.325224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:20.325224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:20.325224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:20.325224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:20.325224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:20.325224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:20.325224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 05:31:20.325224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:20.325224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:20.325224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:20.325224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:20.325224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:20.325224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:20.325224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:20.325224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:20.326283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:20.326283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:20.326283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:20.326283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:20.326283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:20.326283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:20.326283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:20.326283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:20.326283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:20.326283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:20.326283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:20.326283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:20.326283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:20.326283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:20.326283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:20.326283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:20.327189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:20.327189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:20.327189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:20.327189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:20.327189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:20.327189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:20.327189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:20.327189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:20.327189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:20.327189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:20.327189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:20.327189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:20.327189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 05:31:20.327189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:20.327189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 353/717] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:20 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:20.332504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c37669-0e70-43b5-b01d-9c648233b82a None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:20.462708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c37669-0e70-43b5-b01d-9c648233b82a None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:20.503737 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661533e-8448-47b2-b9de-2e2738092893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:20.504417 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661533e-8448-47b2-b9de-2e2738092893 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:20.504696 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661533e-8448-47b2-b9de-2e2738092893 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:20.504968 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661533e-8448-47b2-b9de-2e2738092893 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:20.505675 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c661533e-8448-47b2-b9de-2e2738092893 None admin] RBAC: Adding query filter params (name=network) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:20.505981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c661533e-8448-47b2-b9de-2e2738092893 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:20.507470 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c661533e-8448-47b2-b9de-2e2738092893 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:20.514655 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 365/718] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jul 21 05:31:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:20.520279 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d819e-b3e1-4b3d-aa92-70eca6f2983a None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:20.649881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d819e-b3e1-4b3d-aa92-70eca6f2983a None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:20.689571 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17625dd3-1cb1-4422-8d6b-421933199862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:20.690215 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17625dd3-1cb1-4422-8d6b-421933199862 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:20.690494 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17625dd3-1cb1-4422-8d6b-421933199862 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:20.690767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17625dd3-1cb1-4422-8d6b-421933199862 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:20.691562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17625dd3-1cb1-4422-8d6b-421933199862 None admin] RBAC: Adding query filter params (type=network) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:20.691860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17625dd3-1cb1-4422-8d6b-421933199862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:20.693232 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17625dd3-1cb1-4422-8d6b-421933199862 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:20.703375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 354/719] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jul 21 05:31:20 2022] GET /identity/v3/services?type=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:20.713472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf63c1-2a92-43d5-892d-3ece6f3876b5 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:20.713814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf63c1-2a92-43d5-892d-3ece6f3876b5 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:20.714032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf63c1-2a92-43d5-892d-3ece6f3876b5 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:20.717091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 366/720] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:20.722064 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec929397-169d-438b-bb6b-7b011dc0f13a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:20.722367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec929397-169d-438b-bb6b-7b011dc0f13a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:20.722641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec929397-169d-438b-bb6b-7b011dc0f13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:20.752290 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-ec929397-169d-438b-bb6b-7b011dc0f13a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:20.753576 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ec929397-169d-438b-bb6b-7b011dc0f13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:20.791781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:20.865705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 355/721] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:31:20.889835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf63c1-2a92-43d5-892d-3ece6f3876b5 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:20.902717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:20.994565 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf63c1-2a92-43d5-892d-3ece6f3876b5 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:21.028523 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77a980-a037-4741-9197-6cbe19bd9a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:21.029336 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77a980-a037-4741-9197-6cbe19bd9a7b None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:21.029623 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77a980-a037-4741-9197-6cbe19bd9a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:21.029900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77a980-a037-4741-9197-6cbe19bd9a7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:21.041136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77a980-a037-4741-9197-6cbe19bd9a7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:21.048978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-df77a980-a037-4741-9197-6cbe19bd9a7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:31:21.048978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:21.048978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:21.048978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:21.048978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:21.048978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:21.048978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:21.048978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:21.048978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:21.048978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:21.048978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:21.048978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:21.048978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:21.048978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:21.048978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:21.048978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:21.049936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:21.049936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:21.049936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:21.049936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:21.049936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:21.049936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:21.049936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:21.049936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:21.049936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:21.049936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:21.049936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:21.049936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:21.049936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:21.049936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:21.049936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:21.049936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:21.050786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:21.050786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:21.050786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:21.050786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:21.050786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:21.050786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:21.050786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:21.050786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:21.050786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:21.050786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:21.050786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:31:21.050786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:21.055146 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 367/722] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:31:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:21.056694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec929397-169d-438b-bb6b-7b011dc0f13a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:21.190807 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec929397-169d-438b-bb6b-7b011dc0f13a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:21.235709 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465e13a-3a33-4b06-92ce-611fd7229504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:21.236392 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465e13a-3a33-4b06-92ce-611fd7229504 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:21.236677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465e13a-3a33-4b06-92ce-611fd7229504 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:21.236952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465e13a-3a33-4b06-92ce-611fd7229504 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:21.237720 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465e13a-3a33-4b06-92ce-611fd7229504 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:21.238031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465e13a-3a33-4b06-92ce-611fd7229504 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:21.239450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465e13a-3a33-4b06-92ce-611fd7229504 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:21.250495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 356/723] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:31:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:21.256222 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77a980-a037-4741-9197-6cbe19bd9a7b None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:21.299345 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afad42-c4a9-4e22-bb22-df30b6686fb7 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:21.299705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afad42-c4a9-4e22-bb22-df30b6686fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:21.299973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afad42-c4a9-4e22-bb22-df30b6686fb7 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:21.301779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 357/724] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:21.307498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f066a-e6ce-440b-8cee-970a7a70cf21 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:21.307787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f066a-e6ce-440b-8cee-970a7a70cf21 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:21.308074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f066a-e6ce-440b-8cee-970a7a70cf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:21.336678 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-595f066a-e6ce-440b-8cee-970a7a70cf21 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:21.337880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-595f066a-e6ce-440b-8cee-970a7a70cf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:21.367405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:21.379839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:21.406541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77a980-a037-4741-9197-6cbe19bd9a7b None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:21.430433 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 358/725] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:31:21.442346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809a340-ea1b-4c60-b2eb-44443bf7ab7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:21.443143 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809a340-ea1b-4c60-b2eb-44443bf7ab7a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:21.443435 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809a340-ea1b-4c60-b2eb-44443bf7ab7a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:21.443713 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809a340-ea1b-4c60-b2eb-44443bf7ab7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:21.455785 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809a340-ea1b-4c60-b2eb-44443bf7ab7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:21.457310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809a340-ea1b-4c60-b2eb-44443bf7ab7a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:21.472145 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f066a-e6ce-440b-8cee-970a7a70cf21 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:21.477013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-5809a340-ea1b-4c60-b2eb-44443bf7ab7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:21.477013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:21.477013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:21.477013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:21.477013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:21.477013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:21.477013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:21.477013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:21.477013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:21.477013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:21.477013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:21.477013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:21.477013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:21.477013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:21.477013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:21.477889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:21.477889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:21.477889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:21.477889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:21.477889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:21.477889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:21.477889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:21.477889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:21.477889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:21.477889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:21.477889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:21.477889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:21.477889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:21.477889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:21.477889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:21.477889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:21.477889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:21.478770 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:21.478770 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:21.478770 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:21.478770 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:21.478770 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 368/726] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:21.485870 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809a340-ea1b-4c60-b2eb-44443bf7ab7a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:21.644018 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f066a-e6ce-440b-8cee-970a7a70cf21 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:21.671091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809a340-ea1b-4c60-b2eb-44443bf7ab7a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:21.694598 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:21.707679 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20b188-23f8-4617-89a8-868a01e0ca6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:21.708499 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20b188-23f8-4617-89a8-868a01e0ca6e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:21.708798 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20b188-23f8-4617-89a8-868a01e0ca6e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:21.709100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20b188-23f8-4617-89a8-868a01e0ca6e None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:21.709827 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20b188-23f8-4617-89a8-868a01e0ca6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:21.710187 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20b188-23f8-4617-89a8-868a01e0ca6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:21.711617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20b188-23f8-4617-89a8-868a01e0ca6e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:21.725019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 369/727] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:21.730406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20b188-23f8-4617-89a8-868a01e0ca6e None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:21.740642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d28bf2-df11-4cdf-bedf-c9066867f8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:21.741537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d28bf2-df11-4cdf-bedf-c9066867f8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:21.741827 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d28bf2-df11-4cdf-bedf-c9066867f8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:21.742103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d28bf2-df11-4cdf-bedf-c9066867f8c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:21.767873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d28bf2-df11-4cdf-bedf-c9066867f8c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:21.775608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-f8d28bf2-df11-4cdf-bedf-c9066867f8c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:21.775608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:21.775608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:21.775608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:21.775608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:21.775608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:21.775608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:21.775608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:21.775608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:21.775608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:21.775608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:21.775608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:21.775608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:21.775608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:21.775608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:21.775608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:21.776514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:21.776514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:21.776514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:21.776514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:21.776514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:21.776514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:21.776514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:21.776514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:21.776514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:21.776514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:21.776514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:21.776514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:21.776514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:21.776514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:21.776514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:21.776514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:21.777358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:21.777358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:21.777358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:21.777358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:21.777358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:21.777358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:21.777358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:21.777358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:21.777358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:21.777358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:21.777358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:21.777358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:21.783496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 359/728] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:31:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:21.789597 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d28bf2-df11-4cdf-bedf-c9066867f8c5 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:21.806118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:21.833503 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20b188-23f8-4617-89a8-868a01e0ca6e None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:21.895352 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62950496-255a-40a3-ac59-70a1e3173239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:21.896166 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62950496-255a-40a3-ac59-70a1e3173239 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:21.896448 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62950496-255a-40a3-ac59-70a1e3173239 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:21.896720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62950496-255a-40a3-ac59-70a1e3173239 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:21.910325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62950496-255a-40a3-ac59-70a1e3173239 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:21.912979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62950496-255a-40a3-ac59-70a1e3173239 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:21.933618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-62950496-255a-40a3-ac59-70a1e3173239 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 05:31:21.933618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:21.933618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:21.933618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:21.933618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:21.933618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:21.933618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:21.933618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:21.933618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:21.933618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:21.933618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:21.933618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:21.933618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:21.933618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:21.933618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:21.933618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:21.935914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:21.938098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:21.938098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:21.938098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:21.938098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:21.938098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:21.938098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:21.938098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:21.938098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:21.938098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 05:31:21.938098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:21.939350 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 370/729] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:31:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:21.943039 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62950496-255a-40a3-ac59-70a1e3173239 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:21.983855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d28bf2-df11-4cdf-bedf-c9066867f8c5 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:21.999476 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:22.043550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62950496-255a-40a3-ac59-70a1e3173239 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:22.061105 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9bd50-da0e-41b4-be51-476d7a45e7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:22.061795 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9bd50-da0e-41b4-be51-476d7a45e7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:22.062094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9bd50-da0e-41b4-be51-476d7a45e7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:22.062345 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9bd50-da0e-41b4-be51-476d7a45e7c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:22.063138 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9bd50-da0e-41b4-be51-476d7a45e7c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:22.063450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9bd50-da0e-41b4-be51-476d7a45e7c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:22.064853 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9bd50-da0e-41b4-be51-476d7a45e7c6 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:22.080118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd56e3-bbfa-46f8-b7de-64065218bbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:22.080828 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd56e3-bbfa-46f8-b7de-64065218bbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:22.081107 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd56e3-bbfa-46f8-b7de-64065218bbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:22.081380 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd56e3-bbfa-46f8-b7de-64065218bbc6 None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:22.082241 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd56e3-bbfa-46f8-b7de-64065218bbc6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:22.082542 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd56e3-bbfa-46f8-b7de-64065218bbc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:22.084093 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd56e3-bbfa-46f8-b7de-64065218bbc6 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:22.087151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 360/730] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:31:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:22.092441 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9bd50-da0e-41b4-be51-476d7a45e7c6 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:22.104102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 371/731] 10.222.0.44 () {56 vars in 1135 bytes} [Thu Jul 21 05:31:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:22.109846 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd56e3-bbfa-46f8-b7de-64065218bbc6 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:22.209215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:22.212520 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd56e3-bbfa-46f8-b7de-64065218bbc6 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:22.225444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:22.248860 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2fa0a-c031-4ac8-9200-50f9e5cc3d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:22.249468 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2fa0a-c031-4ac8-9200-50f9e5cc3d4e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:22.249751 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2fa0a-c031-4ac8-9200-50f9e5cc3d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:22.250028 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2fa0a-c031-4ac8-9200-50f9e5cc3d4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:22.286811 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9bd50-da0e-41b4-be51-476d7a45e7c6 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:22.318141 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe2fa0a-c031-4ac8-9200-50f9e5cc3d4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:22.319658 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe2fa0a-c031-4ac8-9200-50f9e5cc3d4e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:22.328280 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7776f0-ce50-4bfd-9f13-2c35f5d564fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:22.329350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7776f0-ce50-4bfd-9f13-2c35f5d564fe None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:22.329843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7776f0-ce50-4bfd-9f13-2c35f5d564fe None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:22.330310 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7776f0-ce50-4bfd-9f13-2c35f5d564fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:22.341896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7776f0-ce50-4bfd-9f13-2c35f5d564fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:22.343693 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7776f0-ce50-4bfd-9f13-2c35f5d564fe None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:22.360843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-0a7776f0-ce50-4bfd-9f13-2c35f5d564fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:22.360843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:22.360843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:22.360843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:22.360843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:22.360843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:22.360843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:22.360843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:22.360843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:22.360843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:22.360843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:22.360843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:22.360843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:22.360843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:22.360843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:22.361721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:22.361721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:22.361721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:22.361721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:22.361721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:22.361721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:22.361721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:22.361721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:22.361721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:22.361721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:22.361721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:22.361721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:22.361721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:22.361721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:22.361721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:22.361721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:22.361721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:22.362635 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:22.362635 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:22.362635 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:22.362635 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:22.364991 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 361/732] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:22.370729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7776f0-ce50-4bfd-9f13-2c35f5d564fe None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:22.443536 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:22.447357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 372/733] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:31:22.452947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2fa0a-c031-4ac8-9200-50f9e5cc3d4e None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:22.551670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7776f0-ce50-4bfd-9f13-2c35f5d564fe None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:22.577010 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:22.588598 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2fa0a-c031-4ac8-9200-50f9e5cc3d4e None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:22.590146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9d55c-c178-49ae-8009-98bbf637c135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:22.590822 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9d55c-c178-49ae-8009-98bbf637c135 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:22.591131 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9d55c-c178-49ae-8009-98bbf637c135 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:22.591413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9d55c-c178-49ae-8009-98bbf637c135 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:22.592140 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9d55c-c178-49ae-8009-98bbf637c135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:22.592448 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9d55c-c178-49ae-8009-98bbf637c135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:22.593848 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9d55c-c178-49ae-8009-98bbf637c135 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:22.605861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 362/734] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:22.607345 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b18521-821b-45c1-b2ce-cd529fce9d28 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:22.607482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b18521-821b-45c1-b2ce-cd529fce9d28 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:22.607695 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b18521-821b-45c1-b2ce-cd529fce9d28 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:22.612996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 363/735] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:22.614172 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3eb14b-1554-41cb-ac03-b3f6ffbbf952 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:22.614453 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3eb14b-1554-41cb-ac03-b3f6ffbbf952 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:22.614726 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3eb14b-1554-41cb-ac03-b3f6ffbbf952 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:22.616178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 364/736] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:22.616702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3eb14b-1554-41cb-ac03-b3f6ffbbf952 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:22.627401 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa2dcb-252e-4a21-88c6-d20d13834f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:22.628498 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa2dcb-252e-4a21-88c6-d20d13834f1d None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:22.628953 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa2dcb-252e-4a21-88c6-d20d13834f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:22.629381 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa2dcb-252e-4a21-88c6-d20d13834f1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:22.641223 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa2dcb-252e-4a21-88c6-d20d13834f1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:22.642760 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa2dcb-252e-4a21-88c6-d20d13834f1d None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:22.662157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-c7aa2dcb-252e-4a21-88c6-d20d13834f1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:22.662157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:22.662157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:22.662157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:22.662157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:22.662157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:22.662157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:22.662157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:22.662157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:22.662157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:22.662157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:22.662157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:22.662157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:22.662157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:22.662157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:22.663835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:22.663835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:22.663835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:22.663835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:22.663835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:22.663835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:22.663835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:22.663835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:22.663835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:22.663835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:22.663835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:22.663835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:22.663835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:22.663835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:22.663835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:22.663835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:22.663835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:22.669526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:22.669526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:22.669526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:22.669526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:22.669526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 373/737] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:22.669526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1839dc1-6fd5-476d-aa15-33248fea5bed None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:22.669526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1839dc1-6fd5-476d-aa15-33248fea5bed None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:22.669526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1839dc1-6fd5-476d-aa15-33248fea5bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:22.685657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-b1839dc1-6fd5-476d-aa15-33248fea5bed None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:22.686822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b1839dc1-6fd5-476d-aa15-33248fea5bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:22.732520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3eb14b-1554-41cb-ac03-b3f6ffbbf952 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:22.735483 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:22.756100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 374/738] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:31:22.757274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340591d6-eb4d-4d09-b907-515a5e56e519 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:22.757561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340591d6-eb4d-4d09-b907-515a5e56e519 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:22.757830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340591d6-eb4d-4d09-b907-515a5e56e519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:22.780243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-340591d6-eb4d-4d09-b907-515a5e56e519 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:22.781407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-340591d6-eb4d-4d09-b907-515a5e56e519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:22.781744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575dfa4-ff73-401b-a776-902741c1b025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:22.782578 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575dfa4-ff73-401b-a776-902741c1b025 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:22.782863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575dfa4-ff73-401b-a776-902741c1b025 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:22.783192 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575dfa4-ff73-401b-a776-902741c1b025 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:22.791479 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575dfa4-ff73-401b-a776-902741c1b025 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:22.792974 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575dfa4-ff73-401b-a776-902741c1b025 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:22.809449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-3575dfa4-ff73-401b-a776-902741c1b025 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 05:31:22.809449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:22.809449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:22.809449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:22.809449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:22.809449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:22.809449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:22.809449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:22.809449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:22.809449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:22.809449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:22.809449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:22.809449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:22.809449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:22.809449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:22.809449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:22.810612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:22.811639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:22.811639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:22.811639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:22.811639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:22.811639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:22.811639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:22.811639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:22.811639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:22.811639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 05:31:22.811639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:22.811639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 365/739] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jul 21 05:31:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:22.812246 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575dfa4-ff73-401b-a776-902741c1b025 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:22.857773 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:22.859162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 375/740] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:31:22.864085 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340591d6-eb4d-4d09-b907-515a5e56e519 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:22.921355 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575dfa4-ff73-401b-a776-902741c1b025 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:22.958511 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9b0d2-71ee-4528-9e68-0d2407928258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:22.959221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9b0d2-71ee-4528-9e68-0d2407928258 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:22.959498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9b0d2-71ee-4528-9e68-0d2407928258 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:22.959764 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9b0d2-71ee-4528-9e68-0d2407928258 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:22.960596 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9b0d2-71ee-4528-9e68-0d2407928258 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:22.960898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9b0d2-71ee-4528-9e68-0d2407928258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:22.962268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9b0d2-71ee-4528-9e68-0d2407928258 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:22.974185 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 366/741] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:22.975003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9b0d2-71ee-4528-9e68-0d2407928258 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:22.975850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340591d6-eb4d-4d09-b907-515a5e56e519 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:22.991275 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:23.013614 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cebbe-e53d-40d2-9ed3-ddb35f795d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:23.014292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cebbe-e53d-40d2-9ed3-ddb35f795d6f None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:23.014572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cebbe-e53d-40d2-9ed3-ddb35f795d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:23.015117 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cebbe-e53d-40d2-9ed3-ddb35f795d6f None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:23.015588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28cebbe-e53d-40d2-9ed3-ddb35f795d6f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:23.017193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28cebbe-e53d-40d2-9ed3-ddb35f795d6f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:23.033490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 376/742] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jul 21 05:31:22 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:31:23.034131 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cebbe-e53d-40d2-9ed3-ddb35f795d6f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:23.110969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9b0d2-71ee-4528-9e68-0d2407928258 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:23.113277 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:23.157076 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cebbe-e53d-40d2-9ed3-ddb35f795d6f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:23.163566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086583e-ee4a-48e2-92cb-3e5a25f5cce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:23.164270 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086583e-ee4a-48e2-92cb-3e5a25f5cce4 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:23.164555 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086583e-ee4a-48e2-92cb-3e5a25f5cce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:23.164833 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086583e-ee4a-48e2-92cb-3e5a25f5cce4 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:23.165749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086583e-ee4a-48e2-92cb-3e5a25f5cce4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:23.166071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086583e-ee4a-48e2-92cb-3e5a25f5cce4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:23.167650 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086583e-ee4a-48e2-92cb-3e5a25f5cce4 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:23.191193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 367/743] 10.222.0.44 () {56 vars in 1149 bytes} [Thu Jul 21 05:31:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:23.191193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086583e-ee4a-48e2-92cb-3e5a25f5cce4 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:23.200996 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b72ad-a68c-4f82-aa20-31eb0ac60c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:23.201816 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b72ad-a68c-4f82-aa20-31eb0ac60c18 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:23.202102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b72ad-a68c-4f82-aa20-31eb0ac60c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:23.202376 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b72ad-a68c-4f82-aa20-31eb0ac60c18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:23.214046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3b72ad-a68c-4f82-aa20-31eb0ac60c18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:23.221888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-2a3b72ad-a68c-4f82-aa20-31eb0ac60c18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:31:23.221888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:23.221888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:23.221888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:23.221888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:23.221888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:23.221888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:23.221888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:23.221888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:23.221888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:23.221888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:23.221888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:23.221888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:23.221888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:23.221888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:23.221888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:23.222898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:23.222898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:23.222898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:23.222898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:23.222898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:23.222898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:23.222898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:23.222898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:23.222898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:23.222898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:23.222898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:23.222898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:23.222898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:23.222898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:23.222898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:23.222898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:23.223793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:23.223793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:23.223793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:23.223793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:23.223793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:23.223793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:23.223793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:23.223793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:23.223793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:23.223793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:23.223793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:31:23.223793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:23.223793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 377/744] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:31:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:23.224788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17a81c-719f-446c-84f5-4233003fcbeb None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:23.224993 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17a81c-719f-446c-84f5-4233003fcbeb None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:23.225265 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17a81c-719f-446c-84f5-4233003fcbeb None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:23.228817 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 378/745] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:23.230036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17a81c-719f-446c-84f5-4233003fcbeb None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:23.263874 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:23.301301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086583e-ee4a-48e2-92cb-3e5a25f5cce4 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:23.335273 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b2fb8-c7ca-417f-8d7e-ade189f144ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:23.336084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b2fb8-c7ca-417f-8d7e-ade189f144ea None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:23.336374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b2fb8-c7ca-417f-8d7e-ade189f144ea None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:23.336640 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b2fb8-c7ca-417f-8d7e-ade189f144ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:23.343462 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b2fb8-c7ca-417f-8d7e-ade189f144ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:23.343917 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17a81c-719f-446c-84f5-4233003fcbeb None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:23.344929 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b2fb8-c7ca-417f-8d7e-ade189f144ea None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:23.359058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-322b2fb8-c7ca-417f-8d7e-ade189f144ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:23.359058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:23.359058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:23.359058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:23.359058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:23.359058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:23.359058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:23.359058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:23.359058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:23.359058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:23.359058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:23.359058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:23.359058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:23.359058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:23.359058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:23.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:23.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:23.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:23.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:23.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:23.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:23.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:23.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:23.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:23.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:23.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:23.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:23.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:23.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:23.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:23.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:23.360067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:23.360967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:23.360967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:23.360967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:23.360967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:23.360967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:23.360967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:23.360967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:23.360967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:23.360967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 368/746] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:23.361540 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b2fb8-c7ca-417f-8d7e-ade189f144ea None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:23.383635 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:23.385318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1213ad0-b3f6-428e-95aa-12b0345cdabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:23.385986 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1213ad0-b3f6-428e-95aa-12b0345cdabf None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:23.386316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1213ad0-b3f6-428e-95aa-12b0345cdabf None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:23.386591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1213ad0-b3f6-428e-95aa-12b0345cdabf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:23.456542 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1213ad0-b3f6-428e-95aa-12b0345cdabf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:23.458322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1213ad0-b3f6-428e-95aa-12b0345cdabf None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:23.467449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b2fb8-c7ca-417f-8d7e-ade189f144ea None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:23.488371 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:23.504265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0897a30-2b00-42bd-8747-ade6e2ae0a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:23.505099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0897a30-2b00-42bd-8747-ade6e2ae0a47 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:23.505384 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0897a30-2b00-42bd-8747-ade6e2ae0a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:23.505660 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0897a30-2b00-42bd-8747-ade6e2ae0a47 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:23.506410 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0897a30-2b00-42bd-8747-ade6e2ae0a47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:23.506716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0897a30-2b00-42bd-8747-ade6e2ae0a47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:23.508259 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0897a30-2b00-42bd-8747-ade6e2ae0a47 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:23.519045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 369/747] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:31:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:23.519045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba3d63-d595-40b7-8bf6-d614cfdcbedc None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:23.519045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba3d63-d595-40b7-8bf6-d614cfdcbedc None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:23.519045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba3d63-d595-40b7-8bf6-d614cfdcbedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:23.535104 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-50ba3d63-d595-40b7-8bf6-d614cfdcbedc None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:23.536430 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-50ba3d63-d595-40b7-8bf6-d614cfdcbedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:23.573119 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 379/748] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:23 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 21 05:31:23.574327 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1213ad0-b3f6-428e-95aa-12b0345cdabf None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:23.617175 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 370/749] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:23 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 21 05:31:23.618022 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba3d63-d595-40b7-8bf6-d614cfdcbedc None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:23.628329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:23.702349 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1213ad0-b3f6-428e-95aa-12b0345cdabf None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:23.721907 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba3d63-d595-40b7-8bf6-d614cfdcbedc None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:23.730445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:23.739766 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a69125-9efd-4653-a1c2-0081c443a1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:23.740441 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a69125-9efd-4653-a1c2-0081c443a1ba None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:23.740721 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a69125-9efd-4653-a1c2-0081c443a1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:23.740994 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a69125-9efd-4653-a1c2-0081c443a1ba None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:23.742267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a69125-9efd-4653-a1c2-0081c443a1ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:23.742267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a69125-9efd-4653-a1c2-0081c443a1ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:23.743666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a69125-9efd-4653-a1c2-0081c443a1ba None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:23.755864 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 380/750] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:23.757085 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a69125-9efd-4653-a1c2-0081c443a1ba None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:23.761728 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c889db1-3216-416c-8b6a-6ae64204f132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:23.762609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c889db1-3216-416c-8b6a-6ae64204f132 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:23.762896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c889db1-3216-416c-8b6a-6ae64204f132 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:23.763193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c889db1-3216-416c-8b6a-6ae64204f132 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:23.774012 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c889db1-3216-416c-8b6a-6ae64204f132 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:23.775504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c889db1-3216-416c-8b6a-6ae64204f132 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:23.792964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-1c889db1-3216-416c-8b6a-6ae64204f132 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:23.792964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:23.792964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:23.792964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:23.792964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:23.792964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:23.792964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:23.792964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:23.792964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:23.792964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:23.792964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:23.792964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:23.792964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:23.792964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:23.792964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:23.793896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:23.793896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:23.793896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:23.793896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:23.793896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:23.793896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:23.793896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:23.793896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:23.793896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:23.793896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:23.793896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:23.793896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:23.793896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:23.793896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:23.793896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:23.793896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:23.793896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:23.794787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:23.794787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:23.794787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:23.794787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:23.794787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 371/751] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:23.795101 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c889db1-3216-416c-8b6a-6ae64204f132 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:23.860090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:23.872315 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a69125-9efd-4653-a1c2-0081c443a1ba None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:23.905202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c889db1-3216-416c-8b6a-6ae64204f132 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:23.908478 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e833aa3-6a6c-4aa1-a296-bbdf5205e95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:23.909288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e833aa3-6a6c-4aa1-a296-bbdf5205e95a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:23.909572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e833aa3-6a6c-4aa1-a296-bbdf5205e95a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:23.909847 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e833aa3-6a6c-4aa1-a296-bbdf5205e95a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:23.920919 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e833aa3-6a6c-4aa1-a296-bbdf5205e95a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:23.922373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e833aa3-6a6c-4aa1-a296-bbdf5205e95a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:23.939473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-0e833aa3-6a6c-4aa1-a296-bbdf5205e95a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:23.939473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:23.939473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:23.939473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:23.939473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:23.939473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:23.939473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:23.939473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:23.939473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:23.939473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:23.939473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:23.939473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:23.939473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:23.939473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:23.939473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:23.940559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:23.940559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:23.940559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:23.940559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:23.940559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:23.940559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:23.940559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:23.940559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:23.940559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:23.940559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:23.940559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:23.940559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:23.940559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:23.940559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:23.940559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:23.940559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:23.940559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:23.941681 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:23.941681 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:23.941681 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:23.941681 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:23.943730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2c2f0-9cd8-441c-b2c3-73661b775d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:23.944615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2c2f0-9cd8-441c-b2c3-73661b775d10 None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:23.945051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2c2f0-9cd8-441c-b2c3-73661b775d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:23.945377 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 381/752] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:23.945621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2c2f0-9cd8-441c-b2c3-73661b775d10 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:23.946217 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e833aa3-6a6c-4aa1-a296-bbdf5205e95a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:23.946671 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2c2f0-9cd8-441c-b2c3-73661b775d10 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:23.948236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2c2f0-9cd8-441c-b2c3-73661b775d10 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:23.966561 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 372/753] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jul 21 05:31:23 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:31:23.967618 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2c2f0-9cd8-441c-b2c3-73661b775d10 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:24.002058 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:24.078102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e833aa3-6a6c-4aa1-a296-bbdf5205e95a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:24.100848 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2c2f0-9cd8-441c-b2c3-73661b775d10 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:24.118802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:24.119223 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf7c71-d2ce-42c0-a862-42f1f0a5af4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:24.121171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf7c71-d2ce-42c0-a862-42f1f0a5af4a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:24.121727 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf7c71-d2ce-42c0-a862-42f1f0a5af4a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:24.122277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf7c71-d2ce-42c0-a862-42f1f0a5af4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:24.124199 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf7c71-d2ce-42c0-a862-42f1f0a5af4a None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=61b61188b5b94a51ba1ae1a88b78b89f) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:24.124793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf7c71-d2ce-42c0-a862-42f1f0a5af4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:24.127366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf7c71-d2ce-42c0-a862-42f1f0a5af4a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:24.134223 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 382/754] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:23 2022] GET /identity/v3/role_assignments?user.id=61b61188b5b94a51ba1ae1a88b78b89f&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:24.135491 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf7c71-d2ce-42c0-a862-42f1f0a5af4a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:24.164250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfe4fe-55dc-42c7-99f9-497fcce9fb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:24.165219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfe4fe-55dc-42c7-99f9-497fcce9fb73 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:24.165608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfe4fe-55dc-42c7-99f9-497fcce9fb73 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:24.165985 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfe4fe-55dc-42c7-99f9-497fcce9fb73 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:24.167665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dfe4fe-55dc-42c7-99f9-497fcce9fb73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:24.167982 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dfe4fe-55dc-42c7-99f9-497fcce9fb73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:24.169746 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dfe4fe-55dc-42c7-99f9-497fcce9fb73 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:24.202524 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 373/755] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:24.204019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfe4fe-55dc-42c7-99f9-497fcce9fb73 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:24.284162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf7c71-d2ce-42c0-a862-42f1f0a5af4a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:24.299470 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:24.336904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3f9c3-5d13-43cb-bd72-9bb02ba91396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:24.337734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3f9c3-5d13-43cb-bd72-9bb02ba91396 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:24.337853 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfe4fe-55dc-42c7-99f9-497fcce9fb73 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:24.338201 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3f9c3-5d13-43cb-bd72-9bb02ba91396 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:24.338569 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3f9c3-5d13-43cb-bd72-9bb02ba91396 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:24.339633 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3f9c3-5d13-43cb-bd72-9bb02ba91396 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:24.340043 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3f9c3-5d13-43cb-bd72-9bb02ba91396 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:24.341523 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3f9c3-5d13-43cb-bd72-9bb02ba91396 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:24.373789 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 383/756] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:24.374363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510771d-a165-41e3-8536-5d6046735670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:24.375356 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510771d-a165-41e3-8536-5d6046735670 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:24.375826 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510771d-a165-41e3-8536-5d6046735670 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:24.376234 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510771d-a165-41e3-8536-5d6046735670 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:24.378720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3f9c3-5d13-43cb-bd72-9bb02ba91396 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:24.391002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510771d-a165-41e3-8536-5d6046735670 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:24.392779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510771d-a165-41e3-8536-5d6046735670 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:24.423160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-c510771d-a165-41e3-8536-5d6046735670 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:31:24.423160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:24.423160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:24.423160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:24.423160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:24.423160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:24.423160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:24.423160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:24.423160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:24.423160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:24.423160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:24.423160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:24.423160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:24.423160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:24.423160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:24.423160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:24.424562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:24.425716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:24.425716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:24.425716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:24.425716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:24.425716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:24.425716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:24.425716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:24.425716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:24.425716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:31:24.425716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:24.435156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 374/757] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jul 21 05:31:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:24.439697 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510771d-a165-41e3-8536-5d6046735670 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:24.500396 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:24.517141 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3f9c3-5d13-43cb-bd72-9bb02ba91396 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:24.555906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc10781-b2c7-4a48-949c-1f7a3d24d2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:24.556998 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc10781-b2c7-4a48-949c-1f7a3d24d2ed None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:24.557490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc10781-b2c7-4a48-949c-1f7a3d24d2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:24.557901 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc10781-b2c7-4a48-949c-1f7a3d24d2ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:24.566473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc10781-b2c7-4a48-949c-1f7a3d24d2ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:24.568167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc10781-b2c7-4a48-949c-1f7a3d24d2ed None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:24.584915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-9cc10781-b2c7-4a48-949c-1f7a3d24d2ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:24.584915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:24.584915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:24.584915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:24.584915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:24.584915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:24.584915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:24.584915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:24.584915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:24.584915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:24.584915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:24.584915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:24.584915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:24.584915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:24.584915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:24.586102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:24.586102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:24.586102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:24.586102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:24.586102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:24.586102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:24.586102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:24.586102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:24.586102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:24.586102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:24.586102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:24.586102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:24.586102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:24.586102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:24.586102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:24.586102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:24.586102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:24.587229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:24.587229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:24.587229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:24.587229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:24.587229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:24.587229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:24.587229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:24.587229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:24.589683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 384/758] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:24.594608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc10781-b2c7-4a48-949c-1f7a3d24d2ed None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:24.623870 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:24.645981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510771d-a165-41e3-8536-5d6046735670 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:24.710341 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc10781-b2c7-4a48-949c-1f7a3d24d2ed None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:24.746650 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3ce96-6c84-4a9b-97e7-71285b8524e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:24.747642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3ce96-6c84-4a9b-97e7-71285b8524e2 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:24.748404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3ce96-6c84-4a9b-97e7-71285b8524e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:24.748822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3ce96-6c84-4a9b-97e7-71285b8524e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:24.749749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3ce96-6c84-4a9b-97e7-71285b8524e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:24.750213 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3ce96-6c84-4a9b-97e7-71285b8524e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:24.751969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3ce96-6c84-4a9b-97e7-71285b8524e2 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:24.767043 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 385/759] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:24.779212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3ce96-6c84-4a9b-97e7-71285b8524e2 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:24.783137 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe1254-9899-472e-a918-e9ddef4032a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:24.783964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe1254-9899-472e-a918-e9ddef4032a4 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:24.784257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe1254-9899-472e-a918-e9ddef4032a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:24.784539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe1254-9899-472e-a918-e9ddef4032a4 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:24.785412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe1254-9899-472e-a918-e9ddef4032a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:24.785727 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe1254-9899-472e-a918-e9ddef4032a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:24.787241 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe1254-9899-472e-a918-e9ddef4032a4 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:24.837495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 375/760] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jul 21 05:31:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:24.842869 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe1254-9899-472e-a918-e9ddef4032a4 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:24.871650 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:24.890528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3ce96-6c84-4a9b-97e7-71285b8524e2 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:24.932698 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5408a-3570-485b-a511-2e3a24869e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:24.934403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5408a-3570-485b-a511-2e3a24869e06 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:24.939764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5408a-3570-485b-a511-2e3a24869e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:24.940196 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5408a-3570-485b-a511-2e3a24869e06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:24.941173 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5408a-3570-485b-a511-2e3a24869e06 None admin] RBAC: Adding query filter params (role.id=bb94781ab8054924a5cb21b561b12bbb, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=4d04998b636649d98103b1cd5f3a0f7c) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:24.941647 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5408a-3570-485b-a511-2e3a24869e06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:24.943253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b5408a-3570-485b-a511-2e3a24869e06 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:24.953438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 386/761] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:24 2022] GET /identity/v3/role_assignments?user.id=4d04998b636649d98103b1cd5f3a0f7c&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=bb94781ab8054924a5cb21b561b12bbb => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:25.008721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:25.016308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe1254-9899-472e-a918-e9ddef4032a4 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:25.066682 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7998d-4031-4005-bf75-a3e8367c752f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:25.068079 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7998d-4031-4005-bf75-a3e8367c752f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:25.068518 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7998d-4031-4005-bf75-a3e8367c752f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:25.068928 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7998d-4031-4005-bf75-a3e8367c752f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:25.086018 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7998d-4031-4005-bf75-a3e8367c752f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:25.087742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7998d-4031-4005-bf75-a3e8367c752f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:25.113139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-a1f7998d-4031-4005-bf75-a3e8367c752f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:25.113139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:25.113139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:25.113139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:25.113139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:25.113139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:25.113139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:25.113139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:25.113139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:25.113139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:25.113139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:25.113139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:25.113139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:25.113139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:25.113139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:25.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:25.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:25.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:25.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:25.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:25.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:25.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:25.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:25.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:25.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:25.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:25.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:25.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:25.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:25.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:25.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:25.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:25.115456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:25.115456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:25.115456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:25.115456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:25.117105 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 376/762] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:25.123186 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5408a-3570-485b-a511-2e3a24869e06 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:25.200458 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785a0c1-a8c2-4a8e-9a30-620c8747d336 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:25.201334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785a0c1-a8c2-4a8e-9a30-620c8747d336 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:25.201894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785a0c1-a8c2-4a8e-9a30-620c8747d336 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:25.203827 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 377/763] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:25.211431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c796e4-7d35-4a12-9285-e6cd7d898432 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:25.212075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c796e4-7d35-4a12-9285-e6cd7d898432 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:25.212489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c796e4-7d35-4a12-9285-e6cd7d898432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:25.232866 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-86c796e4-7d35-4a12-9285-e6cd7d898432 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:25.235417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-86c796e4-7d35-4a12-9285-e6cd7d898432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:25.275805 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5408a-3570-485b-a511-2e3a24869e06 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:25.316337 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fe41c-0981-467a-9e74-e7c31c8a1188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:25.317872 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fe41c-0981-467a-9e74-e7c31c8a1188 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:25.318296 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fe41c-0981-467a-9e74-e7c31c8a1188 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:25.318711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fe41c-0981-467a-9e74-e7c31c8a1188 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:25.319640 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fe41c-0981-467a-9e74-e7c31c8a1188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:25.319896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 378/764] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:25.320247 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fe41c-0981-467a-9e74-e7c31c8a1188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:25.321747 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fe41c-0981-467a-9e74-e7c31c8a1188 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:25.334791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 387/765] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:25.341442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c796e4-7d35-4a12-9285-e6cd7d898432 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:25.358526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fe41c-0981-467a-9e74-e7c31c8a1188 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:25.378761 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:25.462564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:25.486718 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c796e4-7d35-4a12-9285-e6cd7d898432 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:25.528263 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac03e0f-f27e-476a-98be-d7477638c0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:25.529493 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac03e0f-f27e-476a-98be-d7477638c0db None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:25.529997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac03e0f-f27e-476a-98be-d7477638c0db None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:25.530469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac03e0f-f27e-476a-98be-d7477638c0db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:25.544212 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac03e0f-f27e-476a-98be-d7477638c0db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:25.547329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac03e0f-f27e-476a-98be-d7477638c0db None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:25.555235 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fe41c-0981-467a-9e74-e7c31c8a1188 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:25.577062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-4ac03e0f-f27e-476a-98be-d7477638c0db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:25.577062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:25.577062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:25.577062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:25.577062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:25.577062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:25.577062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:25.577062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:25.577062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:25.577062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:25.577062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:25.577062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:25.577062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:25.577062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:25.577062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:25.578041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:25.578041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:25.578041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:25.578041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:25.578041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:25.578041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:25.578041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:25.578041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:25.578041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:25.578041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:25.578041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:25.578041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:25.578041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:25.578041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:25.578041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:25.578041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:25.578041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:25.578963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:25.578963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:25.578963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:25.578963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:25.578963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:25.578963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:25.578963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:25.578963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:25.587590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 379/766] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:25.592204 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac03e0f-f27e-476a-98be-d7477638c0db None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:25.635875 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a9fe3-91b5-4ed6-8274-9aa8f122c4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:25.637356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a9fe3-91b5-4ed6-8274-9aa8f122c4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:25.638065 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a9fe3-91b5-4ed6-8274-9aa8f122c4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:25.638939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a9fe3-91b5-4ed6-8274-9aa8f122c4d3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:25.640081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a9fe3-91b5-4ed6-8274-9aa8f122c4d3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:25.641908 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a9fe3-91b5-4ed6-8274-9aa8f122c4d3 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:25.668348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-683a9fe3-91b5-4ed6-8274-9aa8f122c4d3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 05:31:25.668348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:25.668348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:25.668348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:25.668348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:25.668348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:25.668348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:25.668348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:25.668348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 05:31:25.668348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:25.668348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:25.668348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:25.668348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:25.668348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:25.668348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:25.668348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:25.669295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:25.669295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:25.669295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:25.669295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:25.669295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:25.669295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:25.669295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:25.669295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:25.669295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:25.669295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:25.669295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:25.669295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:25.669295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:25.669295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:25.669295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:25.669295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:25.670136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:25.670136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:25.670136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:25.670136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:25.670136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:25.670136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:25.670136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:25.670136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:25.670136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:25.670136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:25.670136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:25.670136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:25.670136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:25.670136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 05:31:25.670136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:25.670136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 388/767] 10.222.0.44 () {56 vars in 1129 bytes} [Thu Jul 21 05:31:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:25.684160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a9fe3-91b5-4ed6-8274-9aa8f122c4d3 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:25.765204 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac03e0f-f27e-476a-98be-d7477638c0db None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:25.816456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:25.832945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bb67c-736a-4a62-a670-fdb5c3393930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:25.833815 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bb67c-736a-4a62-a670-fdb5c3393930 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:25.833968 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bb67c-736a-4a62-a670-fdb5c3393930 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:25.834314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bb67c-736a-4a62-a670-fdb5c3393930 None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:25.837223 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91bb67c-736a-4a62-a670-fdb5c3393930 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:25.837547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91bb67c-736a-4a62-a670-fdb5c3393930 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:25.841575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91bb67c-736a-4a62-a670-fdb5c3393930 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:25.850102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:25.856257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 380/768] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:25.861948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bb67c-736a-4a62-a670-fdb5c3393930 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:25.875976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a9fe3-91b5-4ed6-8274-9aa8f122c4d3 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:25.967822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193645a-ebc5-4a80-9819-92f97a07385a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:25.968527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193645a-ebc5-4a80-9819-92f97a07385a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:25.968900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193645a-ebc5-4a80-9819-92f97a07385a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:25.969195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193645a-ebc5-4a80-9819-92f97a07385a None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:25.969973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193645a-ebc5-4a80-9819-92f97a07385a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:25.970284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193645a-ebc5-4a80-9819-92f97a07385a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:25.971733 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193645a-ebc5-4a80-9819-92f97a07385a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:25.980570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bb67c-736a-4a62-a670-fdb5c3393930 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:25.982379 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 389/769] 10.222.0.44 () {56 vars in 1124 bytes} [Thu Jul 21 05:31:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:25.987836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193645a-ebc5-4a80-9819-92f97a07385a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:26.016479 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64904cf-3ae9-4d69-923b-c05cee9f8ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:26.017299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64904cf-3ae9-4d69-923b-c05cee9f8ee9 None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:26.017582 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64904cf-3ae9-4d69-923b-c05cee9f8ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:26.017860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64904cf-3ae9-4d69-923b-c05cee9f8ee9 None admin] PATH_INFO: `/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/928d014648cd41d7b58e7e01845eb748/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:26.019858 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64904cf-3ae9-4d69-923b-c05cee9f8ee9 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b6f27b8d9b44d191210a24a9e04041, user_id=928d014648cd41d7b58e7e01845eb748, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:26.021325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64904cf-3ae9-4d69-923b-c05cee9f8ee9 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:26.040644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 381/770] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:31:25 2022] PUT /identity/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/928d014648cd41d7b58e7e01845eb748/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:31:26.140682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193645a-ebc5-4a80-9819-92f97a07385a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:26.199799 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953f49f-2365-48a9-88ab-21de78015d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:26.200513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953f49f-2365-48a9-88ab-21de78015d8f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:26.200802 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953f49f-2365-48a9-88ab-21de78015d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:26.201081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953f49f-2365-48a9-88ab-21de78015d8f None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:26.201804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953f49f-2365-48a9-88ab-21de78015d8f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:26.202112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953f49f-2365-48a9-88ab-21de78015d8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:26.203610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953f49f-2365-48a9-88ab-21de78015d8f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:26.207348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:26.209771 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 390/771] 10.222.0.44 () {56 vars in 1124 bytes} [Thu Jul 21 05:31:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:26.215580 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64904cf-3ae9-4d69-923b-c05cee9f8ee9 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:26.282065 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:26.330366 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64904cf-3ae9-4d69-923b-c05cee9f8ee9 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:26.367428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f4368-2f34-4e06-bcab-ee5cd6fb07c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:26.368250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f4368-2f34-4e06-bcab-ee5cd6fb07c2 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:26.368536 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f4368-2f34-4e06-bcab-ee5cd6fb07c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:26.368809 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f4368-2f34-4e06-bcab-ee5cd6fb07c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:26.369616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f4368-2f34-4e06-bcab-ee5cd6fb07c2 None admin] RBAC: Adding query filter params (interface=public, service_id=cde8d0c747e3415f94be945b91d490a7, region_id=RegionOne) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:26.369930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f4368-2f34-4e06-bcab-ee5cd6fb07c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:26.371337 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f4368-2f34-4e06-bcab-ee5cd6fb07c2 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:26.379581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 382/772] 10.222.0.44 () {56 vars in 1251 bytes} [Thu Jul 21 05:31:26 2022] GET /identity/v3/endpoints?service_id=cde8d0c747e3415f94be945b91d490a7&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:26.390915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953f49f-2365-48a9-88ab-21de78015d8f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:26.410939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c916f0-fdbc-42f0-93ca-2100aa2383df None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:26.411274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c916f0-fdbc-42f0-93ca-2100aa2383df None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:26.411517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c916f0-fdbc-42f0-93ca-2100aa2383df None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:26.413486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 383/773] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:31:26.418931 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdea08-2230-4b80-86bc-fbc9c636ea44 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:26.419245 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdea08-2230-4b80-86bc-fbc9c636ea44 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:26.419522 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bdea08-2230-4b80-86bc-fbc9c636ea44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:26.449593 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-41bdea08-2230-4b80-86bc-fbc9c636ea44 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:26.450841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-41bdea08-2230-4b80-86bc-fbc9c636ea44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:26.564749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953f49f-2365-48a9-88ab-21de78015d8f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:26.592863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 384/774] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:26.610111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5b9a0-58a2-4161-8054-26f64d3c9e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:26.610796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5b9a0-58a2-4161-8054-26f64d3c9e97 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:26.611102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5b9a0-58a2-4161-8054-26f64d3c9e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:26.611382 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5b9a0-58a2-4161-8054-26f64d3c9e97 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:26.612072 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5b9a0-58a2-4161-8054-26f64d3c9e97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:26.613456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5b9a0-58a2-4161-8054-26f64d3c9e97 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:26.614972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdea08-2230-4b80-86bc-fbc9c636ea44 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:26.619746 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 391/775] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jul 21 05:31:26 2022] GET /identity/v3/services => generated 772 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:26.673443 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:26.730797 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bdea08-2230-4b80-86bc-fbc9c636ea44 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:26.773000 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192a920-c138-4a24-ab20-98331c2b3783 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:26.773353 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192a920-c138-4a24-ab20-98331c2b3783 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:26.773575 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192a920-c138-4a24-ab20-98331c2b3783 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:26.776276 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 392/776] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:26.777865 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07fd0f-fd51-4ec6-be05-840e73994d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:26.778942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07fd0f-fd51-4ec6-be05-840e73994d53 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:26.779473 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07fd0f-fd51-4ec6-be05-840e73994d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:26.779948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07fd0f-fd51-4ec6-be05-840e73994d53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:26.791118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bef9f-762f-4faa-9914-d8e2af73be42 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:26.791462 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bef9f-762f-4faa-9914-d8e2af73be42 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:26.791672 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bef9f-762f-4faa-9914-d8e2af73be42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:26.796544 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07fd0f-fd51-4ec6-be05-840e73994d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:26.804705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-ea07fd0f-fd51-4ec6-be05-840e73994d53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:31:26.804705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:26.804705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:26.804705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:26.804705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:26.804705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:26.804705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:26.804705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:26.804705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:26.804705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:26.804705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:26.804705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:26.804705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:26.804705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:26.804705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:26.804705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:26.805688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:26.805688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:26.805688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:26.805688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:26.805688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:26.805688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:26.805688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:26.805688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:26.805688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:26.805688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:26.805688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:26.805688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:26.805688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:26.805688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:26.805688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:26.805688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:26.806523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:26.806523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:26.806523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:26.806523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:26.806523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:26.806523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:26.806523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:26.806523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:26.806523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:26.806523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:26.806523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:31:26.806523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:26.809471 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 385/777] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:31:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:26.816920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07fd0f-fd51-4ec6-be05-840e73994d53 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:26.824592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-970bef9f-762f-4faa-9914-d8e2af73be42 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:26.825778 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-970bef9f-762f-4faa-9914-d8e2af73be42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:26.895253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:26.907143 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 393/778] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:26.930196 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bef9f-762f-4faa-9914-d8e2af73be42 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:26.936127 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07fd0f-fd51-4ec6-be05-840e73994d53 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:26.962905 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:27.041095 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70dbea-ac13-4679-989d-45da594c5eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:27.041775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70dbea-ac13-4679-989d-45da594c5eca None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:27.042057 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70dbea-ac13-4679-989d-45da594c5eca None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:27.042383 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70dbea-ac13-4679-989d-45da594c5eca None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:27.043118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70dbea-ac13-4679-989d-45da594c5eca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:27.043430 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70dbea-ac13-4679-989d-45da594c5eca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:27.044822 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db70dbea-ac13-4679-989d-45da594c5eca None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:27.045280 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bef9f-762f-4faa-9914-d8e2af73be42 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:27.069474 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 386/779] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:31:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:27.075997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70dbea-ac13-4679-989d-45da594c5eca None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:27.089552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673094a-9579-4d05-9da0-ee59f64fccf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:27.090375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673094a-9579-4d05-9da0-ee59f64fccf7 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:27.090649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673094a-9579-4d05-9da0-ee59f64fccf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:27.090969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673094a-9579-4d05-9da0-ee59f64fccf7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:27.091676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673094a-9579-4d05-9da0-ee59f64fccf7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:27.096875 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673094a-9579-4d05-9da0-ee59f64fccf7 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:27.120289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-0673094a-9579-4d05-9da0-ee59f64fccf7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 05:31:27.120289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:27.120289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:27.120289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:27.120289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:27.120289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:27.120289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:27.120289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:27.120289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 05:31:27.120289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:27.120289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:27.120289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:27.120289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:27.120289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:27.120289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:27.120289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:27.120289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:27.121333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:27.121333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:27.121333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:27.121333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:27.121333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:27.121333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:27.121333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:27.121333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:27.121333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:27.121333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:27.121333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:27.121333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:27.121333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:27.121333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:27.121333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:27.121333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:27.122165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:27.122165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:27.122165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:27.122165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:27.122165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:27.122165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:27.122165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:27.122165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:27.122165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:27.122165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:27.122165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:27.122165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:27.122165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 05:31:27.122165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:27.128854 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 394/780] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:26 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:27.142697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673094a-9579-4d05-9da0-ee59f64fccf7 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:27.224930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db70dbea-ac13-4679-989d-45da594c5eca None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:27.255344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:27.258673 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4996bf7-2485-45b4-ae27-28d33dfd5040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:27.259518 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4996bf7-2485-45b4-ae27-28d33dfd5040 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:27.259904 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4996bf7-2485-45b4-ae27-28d33dfd5040 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:27.260202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4996bf7-2485-45b4-ae27-28d33dfd5040 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:27.271335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4996bf7-2485-45b4-ae27-28d33dfd5040 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:27.272868 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4996bf7-2485-45b4-ae27-28d33dfd5040 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:27.290570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-e4996bf7-2485-45b4-ae27-28d33dfd5040 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:27.290570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:27.290570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:27.290570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:27.290570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:27.290570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:27.290570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:27.290570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:27.290570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:27.290570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:27.290570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:27.290570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:27.290570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:27.290570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:27.290570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:27.291553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:27.291553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:27.291553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:27.291553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:27.291553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:27.291553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:27.291553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:27.291553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:27.291553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:27.291553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:27.291553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:27.291553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:27.291553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:27.291553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:27.291553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:27.291553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:27.291553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:27.292427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:27.292427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:27.292427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:27.292427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:27.292427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 387/781] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:27.299199 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4996bf7-2485-45b4-ae27-28d33dfd5040 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:27.318290 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673094a-9579-4d05-9da0-ee59f64fccf7 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:27.346022 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:27.356661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df62d1-3c01-4419-a7ce-fff529296551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:27.357698 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df62d1-3c01-4419-a7ce-fff529296551 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:27.358185 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df62d1-3c01-4419-a7ce-fff529296551 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:27.358655 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df62d1-3c01-4419-a7ce-fff529296551 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:27.359599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df62d1-3c01-4419-a7ce-fff529296551 None admin] RBAC: Adding query filter params (name=network) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:27.360155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df62d1-3c01-4419-a7ce-fff529296551 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:27.361755 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df62d1-3c01-4419-a7ce-fff529296551 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:27.367483 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 395/782] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jul 21 05:31:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:27.373402 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df62d1-3c01-4419-a7ce-fff529296551 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:27.482263 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4996bf7-2485-45b4-ae27-28d33dfd5040 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:27.540954 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d349913-8833-4a7f-9023-97cff28d0dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:27.541634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d349913-8833-4a7f-9023-97cff28d0dbb None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:27.541919 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d349913-8833-4a7f-9023-97cff28d0dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:27.542197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d349913-8833-4a7f-9023-97cff28d0dbb None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:27.542933 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d349913-8833-4a7f-9023-97cff28d0dbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:27.543263 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d349913-8833-4a7f-9023-97cff28d0dbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:27.544654 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d349913-8833-4a7f-9023-97cff28d0dbb None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:27.567471 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 388/783] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:27.568921 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540aae7-0d17-4d88-ba4c-37bcd34d7aaf None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:27.569218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540aae7-0d17-4d88-ba4c-37bcd34d7aaf None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:27.569527 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540aae7-0d17-4d88-ba4c-37bcd34d7aaf None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:27.570978 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 389/784] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:27.572364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540aae7-0d17-4d88-ba4c-37bcd34d7aaf None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:27.577238 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df62d1-3c01-4419-a7ce-fff529296551 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:27.603622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:27.618773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ff082-9049-4711-b8df-a247aa48612e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:27.619490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ff082-9049-4711-b8df-a247aa48612e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:27.620197 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ff082-9049-4711-b8df-a247aa48612e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:27.620476 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ff082-9049-4711-b8df-a247aa48612e None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:27.621467 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ff082-9049-4711-b8df-a247aa48612e None admin] RBAC: Adding query filter params (type=network) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:27.621775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ff082-9049-4711-b8df-a247aa48612e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:27.623923 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ff082-9049-4711-b8df-a247aa48612e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:27.630786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 396/785] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jul 21 05:31:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:27.632608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310767c7-6fca-4df8-b242-b517ac885eae None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:27.632904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310767c7-6fca-4df8-b242-b517ac885eae None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:27.633610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310767c7-6fca-4df8-b242-b517ac885eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:27.655711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-310767c7-6fca-4df8-b242-b517ac885eae None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:27.657359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-310767c7-6fca-4df8-b242-b517ac885eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:27.668282 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:27.685916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540aae7-0d17-4d88-ba4c-37bcd34d7aaf None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:27.724727 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9bd23-1556-47e1-8717-eb888d29e854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:27.725695 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9bd23-1556-47e1-8717-eb888d29e854 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:27.726037 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9bd23-1556-47e1-8717-eb888d29e854 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:27.726381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec9bd23-1556-47e1-8717-eb888d29e854 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:27.733922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9bd23-1556-47e1-8717-eb888d29e854 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:27.735563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec9bd23-1556-47e1-8717-eb888d29e854 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:27.750454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-dec9bd23-1556-47e1-8717-eb888d29e854 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 05:31:27.750454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:27.750454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:27.750454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:27.750454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:27.750454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:27.750454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:27.750454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:27.750454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:27.750454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:27.750454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:27.750454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:27.750454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:27.750454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:27.750454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:27.750454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:27.751608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:27.752830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:27.752830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:27.752830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:27.752830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:27.752830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:27.752830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:27.752830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:27.752830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:27.752830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 05:31:27.752830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:27.752830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 390/787] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:31:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:27.753441 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9bd23-1556-47e1-8717-eb888d29e854 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:27.753554 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 397/787] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:27.758222 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310767c7-6fca-4df8-b242-b517ac885eae None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:27.880988 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310767c7-6fca-4df8-b242-b517ac885eae None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:27.886199 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec9bd23-1556-47e1-8717-eb888d29e854 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:27.898701 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:27.919206 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253f60c-0220-48cb-864a-2cc58681a590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:27.919907 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253f60c-0220-48cb-864a-2cc58681a590 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:27.920187 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253f60c-0220-48cb-864a-2cc58681a590 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:27.920460 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253f60c-0220-48cb-864a-2cc58681a590 None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:27.921517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253f60c-0220-48cb-864a-2cc58681a590 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:27.921828 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253f60c-0220-48cb-864a-2cc58681a590 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:27.923972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253f60c-0220-48cb-864a-2cc58681a590 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:27.931122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478c74b-3392-43b0-8e95-b446b08fec18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:27.932238 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478c74b-3392-43b0-8e95-b446b08fec18 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:27.932667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478c74b-3392-43b0-8e95-b446b08fec18 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:27.933288 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478c74b-3392-43b0-8e95-b446b08fec18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:27.935194 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478c74b-3392-43b0-8e95-b446b08fec18 None admin] RBAC: Adding query filter params (interface=public, service_id=8278175f0dc146808f8d2baceb86aa2f, region_id=RegionOne) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:27.935661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478c74b-3392-43b0-8e95-b446b08fec18 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:27.937189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478c74b-3392-43b0-8e95-b446b08fec18 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:27.943662 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 398/788] 10.222.0.44 () {56 vars in 1135 bytes} [Thu Jul 21 05:31:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:27.944690 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253f60c-0220-48cb-864a-2cc58681a590 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:27.944834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 391/789] 10.222.0.44 () {56 vars in 1251 bytes} [Thu Jul 21 05:31:27 2022] GET /identity/v3/endpoints?service_id=8278175f0dc146808f8d2baceb86aa2f&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:27.949663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478c74b-3392-43b0-8e95-b446b08fec18 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:27.963513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:28.074746 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478c74b-3392-43b0-8e95-b446b08fec18 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:28.078178 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253f60c-0220-48cb-864a-2cc58681a590 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:28.114513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9879025-451a-442e-9413-e055bee9e3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:28.115942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9879025-451a-442e-9413-e055bee9e3aa None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:28.116399 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9879025-451a-442e-9413-e055bee9e3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:28.116814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9879025-451a-442e-9413-e055bee9e3aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:28.119176 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7735b9-4cd8-4113-9e93-951f2c269aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:28.120081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7735b9-4cd8-4113-9e93-951f2c269aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:28.120488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7735b9-4cd8-4113-9e93-951f2c269aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:28.120885 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7735b9-4cd8-4113-9e93-951f2c269aa3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:28.129923 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9879025-451a-442e-9413-e055bee9e3aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:28.131772 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7735b9-4cd8-4113-9e93-951f2c269aa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:28.132189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9879025-451a-442e-9413-e055bee9e3aa None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:28.140792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-bf7735b9-4cd8-4113-9e93-951f2c269aa3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:28.140792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:28.140792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:28.140792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:28.140792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:28.140792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:28.140792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:28.140792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:28.140792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:28.140792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:28.140792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:28.140792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:28.140792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:28.140792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:28.140792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:28.140792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:28.141939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:28.141939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:28.141939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:28.141939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:28.141939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:28.141939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:28.141939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:28.141939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:28.141939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:28.141939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:28.141939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:28.141939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:28.141939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:28.141939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:28.141939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:28.141939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:28.143024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:28.143024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:28.143024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:28.143024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:28.143024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:28.143024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:28.143024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:28.143024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:28.143024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:28.143024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:28.143024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:28.143024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:28.150112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 399/790] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:31:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:28.150112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7735b9-4cd8-4113-9e93-951f2c269aa3 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:28.163767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-b9879025-451a-442e-9413-e055bee9e3aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:28.163767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:28.163767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:28.163767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:28.163767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:28.163767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:28.163767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:28.163767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:28.163767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:28.163767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:28.163767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:28.163767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:28.163767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:28.163767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:28.163767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:28.164740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:28.164740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:28.164740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:28.164740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:28.164740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:28.164740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:28.164740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:28.164740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:28.164740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:28.164740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:28.164740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:28.164740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:28.164740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:28.164740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:28.164740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:28.164740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:28.164740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:28.165685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:28.165685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:28.165685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:28.165685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:28.168122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 392/791] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:28.169641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9879025-451a-442e-9413-e055bee9e3aa None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:28.188889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:28.236476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:28.263796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7735b9-4cd8-4113-9e93-951f2c269aa3 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:28.281560 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9879025-451a-442e-9413-e055bee9e3aa None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:28.298398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250db357-3dbd-450c-ade5-d1c8c7781945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:28.299391 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250db357-3dbd-450c-ade5-d1c8c7781945 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:28.299796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250db357-3dbd-450c-ade5-d1c8c7781945 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:28.300163 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250db357-3dbd-450c-ade5-d1c8c7781945 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:28.301035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250db357-3dbd-450c-ade5-d1c8c7781945 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:28.302673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250db357-3dbd-450c-ade5-d1c8c7781945 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:28.315549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 400/792] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jul 21 05:31:28 2022] GET /identity/v3/services => generated 772 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:28.316654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250db357-3dbd-450c-ade5-d1c8c7781945 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:28.319315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfa16d-8d12-4682-a5ed-e43b4246d490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:28.320251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfa16d-8d12-4682-a5ed-e43b4246d490 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:28.320768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfa16d-8d12-4682-a5ed-e43b4246d490 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:28.321244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfa16d-8d12-4682-a5ed-e43b4246d490 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:28.322205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cfa16d-8d12-4682-a5ed-e43b4246d490 None admin] RBAC: Adding query filter params (name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:28.322700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cfa16d-8d12-4682-a5ed-e43b4246d490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:28.324326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cfa16d-8d12-4682-a5ed-e43b4246d490 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:28.334051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 393/793] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:31:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:28.343094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfa16d-8d12-4682-a5ed-e43b4246d490 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:28.461735 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:28.467938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250db357-3dbd-450c-ade5-d1c8c7781945 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:28.484131 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfa16d-8d12-4682-a5ed-e43b4246d490 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:28.505578 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b52f13-f233-401e-8f2b-b1c564079c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:28.506259 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b52f13-f233-401e-8f2b-b1c564079c8a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:28.506543 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b52f13-f233-401e-8f2b-b1c564079c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:28.506867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b52f13-f233-401e-8f2b-b1c564079c8a None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:28.507633 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b52f13-f233-401e-8f2b-b1c564079c8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:28.508061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b52f13-f233-401e-8f2b-b1c564079c8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:28.509472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b52f13-f233-401e-8f2b-b1c564079c8a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:28.510921 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:28.526098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 401/794] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:28.527480 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172980f-c43c-4b13-9da4-da3a4f6f9099 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:28.527771 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172980f-c43c-4b13-9da4-da3a4f6f9099 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:28.528050 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172980f-c43c-4b13-9da4-da3a4f6f9099 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:28.529526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 402/795] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:28.534403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172980f-c43c-4b13-9da4-da3a4f6f9099 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:28.536482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cd9f8e-555a-46b4-8439-84c62b100d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:28.536482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd9f8e-555a-46b4-8439-84c62b100d42 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:28.536785 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd9f8e-555a-46b4-8439-84c62b100d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:28.537355 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cd9f8e-555a-46b4-8439-84c62b100d42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:28.551381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd9f8e-555a-46b4-8439-84c62b100d42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:28.553505 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cd9f8e-555a-46b4-8439-84c62b100d42 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:28.574468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-86cd9f8e-555a-46b4-8439-84c62b100d42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:28.574468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:28.574468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:28.574468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:28.574468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:28.574468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:28.574468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:28.574468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:28.574468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:28.574468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:28.574468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:28.574468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:28.574468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:28.574468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:28.574468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:28.575456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:28.575456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:28.575456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:28.575456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:28.575456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:28.575456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:28.575456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:28.575456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:28.575456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:28.575456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:28.575456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:28.575456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:28.575456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:28.575456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:28.575456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:28.575456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:28.575456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:28.576374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:28.576374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:28.576374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:28.576374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:28.576844 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 394/796] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:28.578060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8f2a5-7644-4c4e-afde-b126f4a14c4d None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:28.578342 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8f2a5-7644-4c4e-afde-b126f4a14c4d None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:28.578617 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8f2a5-7644-4c4e-afde-b126f4a14c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:28.598554 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-c9a8f2a5-7644-4c4e-afde-b126f4a14c4d None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:28.599954 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c9a8f2a5-7644-4c4e-afde-b126f4a14c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:28.659543 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172980f-c43c-4b13-9da4-da3a4f6f9099 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:28.699046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 395/797] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:28.699046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8f2a5-7644-4c4e-afde-b126f4a14c4d None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:28.723941 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc575cb5-1562-4d55-b4c2-40efcb10476c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:28.725155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc575cb5-1562-4d55-b4c2-40efcb10476c None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:28.725756 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc575cb5-1562-4d55-b4c2-40efcb10476c None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:28.726233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc575cb5-1562-4d55-b4c2-40efcb10476c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:28.738803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc575cb5-1562-4d55-b4c2-40efcb10476c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:28.740647 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc575cb5-1562-4d55-b4c2-40efcb10476c None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:28.755316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-cc575cb5-1562-4d55-b4c2-40efcb10476c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:28.755316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:28.755316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:28.755316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:28.755316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:28.755316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:28.755316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:28.755316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:28.755316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:28.755316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:28.755316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:28.755316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:28.755316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:28.755316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:28.755316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:28.756272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:28.756272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:28.756272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:28.756272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:28.756272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:28.756272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:28.756272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:28.756272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:28.756272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:28.756272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:28.756272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:28.756272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:28.756272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:28.756272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:28.756272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:28.756272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:28.756272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:28.757277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:28.757277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:28.757277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:28.757277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:28.757277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:28.757277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:28.757277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:28.757277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:28.759893 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 403/798] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:28.760909 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc575cb5-1562-4d55-b4c2-40efcb10476c None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:28.784065 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:28.805313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8f2a5-7644-4c4e-afde-b126f4a14c4d None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:28.808002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:28.857822 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96b442-8bf9-48fb-951e-c8344d2e5f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:28.859096 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96b442-8bf9-48fb-951e-c8344d2e5f70 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:28.859952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96b442-8bf9-48fb-951e-c8344d2e5f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:28.860569 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96b442-8bf9-48fb-951e-c8344d2e5f70 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:28.861618 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96b442-8bf9-48fb-951e-c8344d2e5f70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:28.862325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96b442-8bf9-48fb-951e-c8344d2e5f70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:28.864229 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96b442-8bf9-48fb-951e-c8344d2e5f70 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:28.878735 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 396/799] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:28.878911 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc575cb5-1562-4d55-b4c2-40efcb10476c None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:28.879966 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96b442-8bf9-48fb-951e-c8344d2e5f70 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:28.916128 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f3730-3bf8-47e5-b2c5-343a0d17658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:28.917433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f3730-3bf8-47e5-b2c5-343a0d17658c None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:28.917981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f3730-3bf8-47e5-b2c5-343a0d17658c None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:28.918498 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f3730-3bf8-47e5-b2c5-343a0d17658c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:28.930100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f3730-3bf8-47e5-b2c5-343a0d17658c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:28.938588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-2a5f3730-3bf8-47e5-b2c5-343a0d17658c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:31:28.938588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:28.938588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:28.938588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:28.938588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:28.938588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:28.938588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:28.938588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:28.938588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:28.938588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:28.938588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:28.938588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:28.938588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:28.938588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:28.938588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:28.938588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:28.939593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:28.939593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:28.939593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:28.939593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:28.939593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:28.939593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:28.939593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:28.939593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:28.939593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:28.939593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:28.939593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:28.939593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:28.939593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:28.939593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:28.939593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:28.939593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:28.940479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:28.940479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:28.940479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:28.940479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:28.940479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:28.940479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:28.940479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:28.940479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:28.940479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:28.940479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:28.940479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:31:28.940479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:28.948229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 404/800] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:31:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:28.949398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f3730-3bf8-47e5-b2c5-343a0d17658c None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:28.985002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96b442-8bf9-48fb-951e-c8344d2e5f70 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:29.018949 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f18a5-27ae-47fd-adb1-7e5f4feaeb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:29.019655 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f18a5-27ae-47fd-adb1-7e5f4feaeb0d None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:29.019944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f18a5-27ae-47fd-adb1-7e5f4feaeb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:29.020210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f18a5-27ae-47fd-adb1-7e5f4feaeb0d None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:29.021002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f18a5-27ae-47fd-adb1-7e5f4feaeb0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:29.021313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f18a5-27ae-47fd-adb1-7e5f4feaeb0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:29.022825 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f18a5-27ae-47fd-adb1-7e5f4feaeb0d None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:29.031378 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:29.039136 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 397/801] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:29.039136 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f18a5-27ae-47fd-adb1-7e5f4feaeb0d None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:29.059889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f3730-3bf8-47e5-b2c5-343a0d17658c None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:29.065728 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:29.096318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83dc15-4239-42b9-9356-668b67f17f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:29.097225 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83dc15-4239-42b9-9356-668b67f17f6a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:29.097506 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83dc15-4239-42b9-9356-668b67f17f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:29.097777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83dc15-4239-42b9-9356-668b67f17f6a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:29.105682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83dc15-4239-42b9-9356-668b67f17f6a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:29.107151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83dc15-4239-42b9-9356-668b67f17f6a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:29.121496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-fd83dc15-4239-42b9-9356-668b67f17f6a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 05:31:29.121496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:29.121496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:29.121496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:29.121496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:29.121496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:29.121496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:29.121496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:29.121496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:29.121496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:29.121496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:29.121496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:29.121496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:29.121496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:29.121496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:29.121496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:29.122501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:29.123527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:29.123527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:29.123527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:29.123527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:29.123527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:29.123527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:29.123527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:29.123527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:29.123527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 05:31:29.123527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:29.127455 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 405/802] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jul 21 05:31:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:29.129437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf8349-a9bd-4d22-af22-fbd7742b70d8 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:29.129725 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf8349-a9bd-4d22-af22-fbd7742b70d8 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:29.130002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf8349-a9bd-4d22-af22-fbd7742b70d8 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:29.131382 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 406/803] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:29.132034 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf8349-a9bd-4d22-af22-fbd7742b70d8 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:29.150186 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f18a5-27ae-47fd-adb1-7e5f4feaeb0d None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:29.185970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83da48-e25c-4212-b973-0b5b52b65d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:29.186646 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83da48-e25c-4212-b973-0b5b52b65d64 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:29.186931 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83da48-e25c-4212-b973-0b5b52b65d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:29.187236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83da48-e25c-4212-b973-0b5b52b65d64 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:29.187993 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83da48-e25c-4212-b973-0b5b52b65d64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:29.188299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83da48-e25c-4212-b973-0b5b52b65d64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:29.189675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83da48-e25c-4212-b973-0b5b52b65d64 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:29.197922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 398/804] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:31:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:29.198731 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83da48-e25c-4212-b973-0b5b52b65d64 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:29.252541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf8349-a9bd-4d22-af22-fbd7742b70d8 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:29.287111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8b959-3521-43df-9660-049cb1702666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:29.288831 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:29.289678 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8b959-3521-43df-9660-049cb1702666 None admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:29.290159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8b959-3521-43df-9660-049cb1702666 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:29.290627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8b959-3521-43df-9660-049cb1702666 None admin] PATH_INFO: `/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/61b61188b5b94a51ba1ae1a88b78b89f/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:29.292884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8b959-3521-43df-9660-049cb1702666 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b6f27b8d9b44d191210a24a9e04041, user_id=61b61188b5b94a51ba1ae1a88b78b89f, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:29.294527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8b959-3521-43df-9660-049cb1702666 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:29.302801 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:29.305640 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83da48-e25c-4212-b973-0b5b52b65d64 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:29.319040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 407/805] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:31:29 2022] PUT /identity/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/61b61188b5b94a51ba1ae1a88b78b89f/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:31:29.319040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae06ab-1122-4853-b5cb-c07f5ddb92bc None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:29.319040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae06ab-1122-4853-b5cb-c07f5ddb92bc None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:29.319040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae06ab-1122-4853-b5cb-c07f5ddb92bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:29.333705 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-e9ae06ab-1122-4853-b5cb-c07f5ddb92bc None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:29.335121 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-e9ae06ab-1122-4853-b5cb-c07f5ddb92bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:29.349239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc08715-14e7-4509-912c-a8a4e2594f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:29.350096 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc08715-14e7-4509-912c-a8a4e2594f90 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:29.350566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc08715-14e7-4509-912c-a8a4e2594f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:29.350937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc08715-14e7-4509-912c-a8a4e2594f90 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:29.351929 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc08715-14e7-4509-912c-a8a4e2594f90 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:29.352339 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc08715-14e7-4509-912c-a8a4e2594f90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:29.353935 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc08715-14e7-4509-912c-a8a4e2594f90 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:29.373437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 399/806] 10.222.0.44 () {56 vars in 1149 bytes} [Thu Jul 21 05:31:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:29.374612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc08715-14e7-4509-912c-a8a4e2594f90 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:29.423488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 408/807] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:29.424873 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae06ab-1122-4853-b5cb-c07f5ddb92bc None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:29.488752 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc08715-14e7-4509-912c-a8a4e2594f90 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:29.537115 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306bf3b0-f414-4353-9043-50c65a77c2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:29.538366 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bf3b0-f414-4353-9043-50c65a77c2cf None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:29.538648 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bf3b0-f414-4353-9043-50c65a77c2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:29.538918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bf3b0-f414-4353-9043-50c65a77c2cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:29.553903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306bf3b0-f414-4353-9043-50c65a77c2cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:29.556294 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306bf3b0-f414-4353-9043-50c65a77c2cf None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:29.564610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae06ab-1122-4853-b5cb-c07f5ddb92bc None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:29.576668 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:29.581080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-306bf3b0-f414-4353-9043-50c65a77c2cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:29.581080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:29.581080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:29.581080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:29.581080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:29.581080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:29.581080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:29.581080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:29.581080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:29.581080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:29.581080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:29.581080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:29.581080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:29.581080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:29.581080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:29.582071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:29.582071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:29.582071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:29.582071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:29.582071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:29.582071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:29.582071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:29.582071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:29.582071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:29.582071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:29.582071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:29.582071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:29.582071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:29.582071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:29.582071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:29.582071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:29.582071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:29.583267 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:29.583267 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:29.583267 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:29.583267 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:29.583267 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 400/808] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:29.583860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306bf3b0-f414-4353-9043-50c65a77c2cf None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:29.599510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:29.608317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf4365-c2bd-412d-b155-0ad48a5ce5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:29.609422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf4365-c2bd-412d-b155-0ad48a5ce5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:29.609905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf4365-c2bd-412d-b155-0ad48a5ce5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:29.610378 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf4365-c2bd-412d-b155-0ad48a5ce5c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:29.626229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf4365-c2bd-412d-b155-0ad48a5ce5c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:29.628164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bf4365-c2bd-412d-b155-0ad48a5ce5c9 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:29.654545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-71bf4365-c2bd-412d-b155-0ad48a5ce5c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:29.654545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:29.654545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:29.654545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:29.654545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:29.654545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:29.654545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:29.654545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:29.654545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:29.654545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:29.654545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:29.654545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:29.654545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:29.654545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:29.654545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:29.655549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:29.655549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:29.655549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:29.655549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:29.655549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:29.655549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:29.655549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:29.655549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:29.655549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:29.655549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:29.655549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:29.655549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:29.655549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:29.655549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:29.655549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:29.655549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:29.655549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:29.656559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:29.656559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:29.656559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:29.656559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:29.659064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 409/809] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:29.660132 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf4365-c2bd-412d-b155-0ad48a5ce5c9 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:29.710877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306bf3b0-f414-4353-9043-50c65a77c2cf None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:29.750783 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ac026-0fc7-434b-8e2d-8462373bfcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:29.751636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ac026-0fc7-434b-8e2d-8462373bfcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:29.751917 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ac026-0fc7-434b-8e2d-8462373bfcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:29.752188 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ac026-0fc7-434b-8e2d-8462373bfcb7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:29.752845 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ac026-0fc7-434b-8e2d-8462373bfcb7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:29.754232 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ac026-0fc7-434b-8e2d-8462373bfcb7 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:29.766381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-e54ac026-0fc7-434b-8e2d-8462373bfcb7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 05:31:29.766381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:29.766381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:29.766381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:29.766381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:29.766381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:29.766381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:29.766381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:29.766381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 05:31:29.766381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:29.766381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:29.766381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:29.766381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:29.766381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:29.766381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:29.766381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:29.767352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:29.767352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:29.767352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:29.767352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:29.767352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:29.767352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:29.767352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:29.767352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:29.767352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:29.767352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:29.767352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:29.767352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:29.767352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:29.767352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:29.767352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:29.767352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:29.768227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:29.768227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:29.768227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:29.768227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:29.768227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:29.768227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:29.768227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:29.768227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:29.768227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:29.768227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:29.768227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:29.768227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:29.768227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:29.768227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 21 05:31:29.768227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:29.768227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 401/810] 10.222.0.44 () {56 vars in 1129 bytes} [Thu Jul 21 05:31:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:29.769092 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ac026-0fc7-434b-8e2d-8462373bfcb7 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:29.769171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf4365-c2bd-412d-b155-0ad48a5ce5c9 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:29.807778 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06c65a-1aae-4678-9070-134b40f66e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:29.808861 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06c65a-1aae-4678-9070-134b40f66e73 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:29.809425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06c65a-1aae-4678-9070-134b40f66e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:29.809994 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06c65a-1aae-4678-9070-134b40f66e73 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:29.810987 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d06c65a-1aae-4678-9070-134b40f66e73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:29.811524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d06c65a-1aae-4678-9070-134b40f66e73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:29.813144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d06c65a-1aae-4678-9070-134b40f66e73 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:29.830605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 410/811] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:29.831859 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06c65a-1aae-4678-9070-134b40f66e73 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:29.833607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:29.878285 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ac026-0fc7-434b-8e2d-8462373bfcb7 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:29.879350 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:29.917883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4828b4d-3eb3-45ea-b350-4d2de57b9a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:29.918924 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4828b4d-3eb3-45ea-b350-4d2de57b9a28 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:29.919391 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4828b4d-3eb3-45ea-b350-4d2de57b9a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:29.919916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4828b4d-3eb3-45ea-b350-4d2de57b9a28 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:29.920845 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4828b4d-3eb3-45ea-b350-4d2de57b9a28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:29.921287 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4828b4d-3eb3-45ea-b350-4d2de57b9a28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:29.922905 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4828b4d-3eb3-45ea-b350-4d2de57b9a28 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:29.942971 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06c65a-1aae-4678-9070-134b40f66e73 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:29.982354 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964b753-4d7c-4f1a-9e98-16a1fd54f278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:29.983087 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b753-4d7c-4f1a-9e98-16a1fd54f278 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:29.983385 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b753-4d7c-4f1a-9e98-16a1fd54f278 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:29.983662 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b753-4d7c-4f1a-9e98-16a1fd54f278 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:29.984388 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964b753-4d7c-4f1a-9e98-16a1fd54f278 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:29.984699 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964b753-4d7c-4f1a-9e98-16a1fd54f278 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:29.986242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964b753-4d7c-4f1a-9e98-16a1fd54f278 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:29.996793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 411/812] 10.222.0.44 () {56 vars in 1124 bytes} [Thu Jul 21 05:31:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:29.997834 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964b753-4d7c-4f1a-9e98-16a1fd54f278 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:30.071335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 402/813] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:30.072928 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4828b4d-3eb3-45ea-b350-4d2de57b9a28 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:30.105544 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964b753-4d7c-4f1a-9e98-16a1fd54f278 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:30.108179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:30.142534 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e912b91-b96d-465c-b40a-cf94c0623915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:30.143379 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e912b91-b96d-465c-b40a-cf94c0623915 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:30.143730 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e912b91-b96d-465c-b40a-cf94c0623915 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:30.143947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e912b91-b96d-465c-b40a-cf94c0623915 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:30.153116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e912b91-b96d-465c-b40a-cf94c0623915 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:30.154617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e912b91-b96d-465c-b40a-cf94c0623915 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:30.170414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-4e912b91-b96d-465c-b40a-cf94c0623915 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:31:30.170414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:30.170414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:30.170414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:30.170414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:30.170414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:30.170414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:30.170414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:30.170414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:30.170414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:30.170414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:30.170414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:30.170414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:30.170414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:30.170414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:30.170414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:30.171657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:30.172722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:30.172722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:30.172722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:30.172722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:30.172722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:30.172722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:30.172722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:30.172722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:30.172722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:31:30.172722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:30.172722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 412/814] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jul 21 05:31:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:30.173322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e912b91-b96d-465c-b40a-cf94c0623915 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:30.185947 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4828b4d-3eb3-45ea-b350-4d2de57b9a28 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:30.207659 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:30.222896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd756484-10ac-49cd-a261-a300748da9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:30.223599 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd756484-10ac-49cd-a261-a300748da9fd None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:30.223879 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd756484-10ac-49cd-a261-a300748da9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:30.224178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd756484-10ac-49cd-a261-a300748da9fd None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:30.224959 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd756484-10ac-49cd-a261-a300748da9fd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:30.225266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd756484-10ac-49cd-a261-a300748da9fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:30.226708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd756484-10ac-49cd-a261-a300748da9fd None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:30.236699 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 403/815] 10.222.0.44 () {56 vars in 1124 bytes} [Thu Jul 21 05:31:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:30.236699 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd756484-10ac-49cd-a261-a300748da9fd None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:30.286665 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e912b91-b96d-465c-b40a-cf94c0623915 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:30.323516 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3d33c-cb62-41e9-8e38-89f0a9659b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:30.324569 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3d33c-cb62-41e9-8e38-89f0a9659b80 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:30.325000 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3d33c-cb62-41e9-8e38-89f0a9659b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:30.325422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc3d33c-cb62-41e9-8e38-89f0a9659b80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:30.333186 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3d33c-cb62-41e9-8e38-89f0a9659b80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:30.334904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc3d33c-cb62-41e9-8e38-89f0a9659b80 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:30.335337 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd756484-10ac-49cd-a261-a300748da9fd None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:30.350775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-afc3d33c-cb62-41e9-8e38-89f0a9659b80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:30.350775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:30.350775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:30.350775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:30.350775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:30.350775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:30.350775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:30.350775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:30.350775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:30.350775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:30.350775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:30.350775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:30.350775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:30.350775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:30.350775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:30.351913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:30.351913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:30.351913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:30.351913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:30.351913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:30.351913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:30.351913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:30.351913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:30.351913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:30.351913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:30.351913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:30.351913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:30.351913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:30.351913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:30.351913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:30.351913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:30.351913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:30.352989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:30.352989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:30.352989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:30.352989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:30.352989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:30.352989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:30.352989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:30.352989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:30.355191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 413/816] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:30.356230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3d33c-cb62-41e9-8e38-89f0a9659b80 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:30.375402 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6eeea-c8ae-4fd8-9a25-020591d33758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:30.376202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6eeea-c8ae-4fd8-9a25-020591d33758 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:30.376449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6eeea-c8ae-4fd8-9a25-020591d33758 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:30.376724 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6eeea-c8ae-4fd8-9a25-020591d33758 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:30.377561 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6eeea-c8ae-4fd8-9a25-020591d33758 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:30.377859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6eeea-c8ae-4fd8-9a25-020591d33758 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:30.379317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6eeea-c8ae-4fd8-9a25-020591d33758 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:30.383871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:30.399406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 404/817] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jul 21 05:31:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:30.401441 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6eeea-c8ae-4fd8-9a25-020591d33758 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:30.448205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:30.466625 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc3d33c-cb62-41e9-8e38-89f0a9659b80 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:30.503120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82709ce2-7609-4415-9413-ba7de9d30379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:30.503799 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82709ce2-7609-4415-9413-ba7de9d30379 None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:30.504088 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82709ce2-7609-4415-9413-ba7de9d30379 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:30.504367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82709ce2-7609-4415-9413-ba7de9d30379 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:30.505169 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82709ce2-7609-4415-9413-ba7de9d30379 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:30.506561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82709ce2-7609-4415-9413-ba7de9d30379 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:30.507641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6eeea-c8ae-4fd8-9a25-020591d33758 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:30.530403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 414/818] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jul 21 05:31:30 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:31:30.531618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82709ce2-7609-4415-9413-ba7de9d30379 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:30.554511 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f7c1c-4400-4bb7-a933-9dfd597fa30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:30.555255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f7c1c-4400-4bb7-a933-9dfd597fa30f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:30.555535 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f7c1c-4400-4bb7-a933-9dfd597fa30f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:30.555812 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f7c1c-4400-4bb7-a933-9dfd597fa30f None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:30.556679 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f7c1c-4400-4bb7-a933-9dfd597fa30f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:30.557028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f7c1c-4400-4bb7-a933-9dfd597fa30f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:30.558538 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f7c1c-4400-4bb7-a933-9dfd597fa30f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:30.571948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 405/819] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:30.573334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2bc78-6cfb-4530-909c-0a5cfd3e4adb None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:30.573687 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2bc78-6cfb-4530-909c-0a5cfd3e4adb None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:30.573910 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2bc78-6cfb-4530-909c-0a5cfd3e4adb None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:30.579112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 406/820] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:30.579112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2bc78-6cfb-4530-909c-0a5cfd3e4adb None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:30.651474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:30.679392 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82709ce2-7609-4415-9413-ba7de9d30379 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:30.717191 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:30.723855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2bc78-6cfb-4530-909c-0a5cfd3e4adb None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:30.724318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfa14d-9ff6-4e4a-a612-92a6c696daba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:30.725061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfa14d-9ff6-4e4a-a612-92a6c696daba None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:30.725348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfa14d-9ff6-4e4a-a612-92a6c696daba None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:30.725659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfa14d-9ff6-4e4a-a612-92a6c696daba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:30.771432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05e2e2-c764-4e68-ab9c-c42b5780dfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:30.772538 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05e2e2-c764-4e68-ab9c-c42b5780dfea None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:30.772818 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05e2e2-c764-4e68-ab9c-c42b5780dfea None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:30.773089 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05e2e2-c764-4e68-ab9c-c42b5780dfea None admin] PATH_INFO: `/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/4d04998b636649d98103b1cd5f3a0f7c/roles/bb94781ab8054924a5cb21b561b12bbb` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:30.794134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05e2e2-c764-4e68-ab9c-c42b5780dfea None admin] RBAC: Authorizing `identity:create_grant(project_id=74b6f27b8d9b44d191210a24a9e04041, user_id=4d04998b636649d98103b1cd5f3a0f7c, role_id=bb94781ab8054924a5cb21b561b12bbb)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:30.795649 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05e2e2-c764-4e68-ab9c-c42b5780dfea None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:30.810980 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfa14d-9ff6-4e4a-a612-92a6c696daba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:30.812414 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 407/821] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:31:30 2022] PUT /identity/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/4d04998b636649d98103b1cd5f3a0f7c/roles/bb94781ab8054924a5cb21b561b12bbb => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:31:30.813184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfa14d-9ff6-4e4a-a612-92a6c696daba None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:30.813792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0329f-6c95-4cac-ba3a-781806b2ae5a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:30.819489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0329f-6c95-4cac-ba3a-781806b2ae5a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:30.821202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0329f-6c95-4cac-ba3a-781806b2ae5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:30.855799 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-1ce0329f-6c95-4cac-ba3a-781806b2ae5a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:30.860369 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1ce0329f-6c95-4cac-ba3a-781806b2ae5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:30.985621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:31.008893 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 408/822] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:31.021178 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 415/823] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 05:31:31.026817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0329f-6c95-4cac-ba3a-781806b2ae5a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:31.038925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfa14d-9ff6-4e4a-a612-92a6c696daba None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:31.131127 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:31.166388 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0329f-6c95-4cac-ba3a-781806b2ae5a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:31.216741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfa14d-9ff6-4e4a-a612-92a6c696daba None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:31.233173 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb70e3c-bb05-465e-8259-2b7e7b3f5f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:31.233982 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb70e3c-bb05-465e-8259-2b7e7b3f5f0f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:31.234268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb70e3c-bb05-465e-8259-2b7e7b3f5f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:31.234545 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb70e3c-bb05-465e-8259-2b7e7b3f5f0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:31.257876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb70e3c-bb05-465e-8259-2b7e7b3f5f0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:31.259392 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb70e3c-bb05-465e-8259-2b7e7b3f5f0f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:31.293814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-adb70e3c-bb05-465e-8259-2b7e7b3f5f0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:31.293814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:31.293814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:31.293814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:31.293814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:31.293814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:31.293814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:31.293814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:31.293814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:31.293814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:31.293814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:31.293814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:31.293814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:31.293814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:31.293814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:31.294792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:31.294792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:31.294792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:31.294792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:31.294792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:31.294792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:31.294792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:31.294792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:31.294792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:31.294792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:31.294792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:31.294792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:31.294792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:31.294792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:31.294792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:31.294792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:31.294792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:31.295712 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:31.295712 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:31.295712 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:31.295712 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:31.295938 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 409/824] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:31.300773 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb70e3c-bb05-465e-8259-2b7e7b3f5f0f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:31.310904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04384f6b-cfe1-4a9f-91f0-bfce0b71ea82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:31.311779 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04384f6b-cfe1-4a9f-91f0-bfce0b71ea82 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:31.312066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04384f6b-cfe1-4a9f-91f0-bfce0b71ea82 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:31.312806 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04384f6b-cfe1-4a9f-91f0-bfce0b71ea82 None admin] PATH_INFO: `/v3/services/network` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:31.313573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04384f6b-cfe1-4a9f-91f0-bfce0b71ea82 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:31.315784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04384f6b-cfe1-4a9f-91f0-bfce0b71ea82 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:31.336577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-04384f6b-cfe1-4a9f-91f0-bfce0b71ea82 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 05:31:31.336577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:31.336577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:31.336577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:31.336577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:31.336577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:31.336577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:31.336577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:31.336577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 05:31:31.336577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:31.336577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:31.336577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:31.336577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:31.336577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:31.336577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:31.336577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:31.336577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:31.338793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:31.338793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:31.338793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:31.338793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:31.338793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:31.338793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:31.338793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:31.338793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:31.338793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:31.338793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:31.338793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:31.338793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:31.338793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:31.338793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:31.338793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:31.338793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:31.340763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:31.340763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:31.340763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:31.340763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:31.340763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:31.340763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:31.340763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:31.340763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:31.340763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:31.340763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:31.340763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:31.340763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:31.340763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 21 05:31:31.340763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:31.340763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 416/825] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:31 2022] GET /identity/v3/services/network => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:31.348279 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04384f6b-cfe1-4a9f-91f0-bfce0b71ea82 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:31.430220 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:31.471735 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb70e3c-bb05-465e-8259-2b7e7b3f5f0f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:31.511051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:31.518455 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25565e45-0cb2-4c12-889c-adfb4c03ca53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:31.519296 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25565e45-0cb2-4c12-889c-adfb4c03ca53 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:31.519588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25565e45-0cb2-4c12-889c-adfb4c03ca53 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:31.519887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25565e45-0cb2-4c12-889c-adfb4c03ca53 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:31.520593 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25565e45-0cb2-4c12-889c-adfb4c03ca53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:31.520900 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25565e45-0cb2-4c12-889c-adfb4c03ca53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:31.522306 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25565e45-0cb2-4c12-889c-adfb4c03ca53 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:31.530733 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04384f6b-cfe1-4a9f-91f0-bfce0b71ea82 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:31.539124 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 410/826] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:31.540547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe2139-b240-405d-93bf-48b09bcac7df None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:31.540837 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe2139-b240-405d-93bf-48b09bcac7df None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:31.541117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe2139-b240-405d-93bf-48b09bcac7df None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:31.547195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 411/827] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:31.547195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe2139-b240-405d-93bf-48b09bcac7df None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:31.576102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c188c-021d-4f27-9b87-089be0aaf803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:31.577306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c188c-021d-4f27-9b87-089be0aaf803 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:31.578021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c188c-021d-4f27-9b87-089be0aaf803 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:31.578885 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c188c-021d-4f27-9b87-089be0aaf803 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:31.580162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c188c-021d-4f27-9b87-089be0aaf803 None admin] RBAC: Adding query filter params (name=network) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:31.580766 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c188c-021d-4f27-9b87-089be0aaf803 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:31.582531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c188c-021d-4f27-9b87-089be0aaf803 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:31.588640 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 417/828] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jul 21 05:31:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:31.590616 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed37bc1-441c-4d96-8d2a-6a73c887432d None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:31.591126 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed37bc1-441c-4d96-8d2a-6a73c887432d None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:31.591626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed37bc1-441c-4d96-8d2a-6a73c887432d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:31.613628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-aed37bc1-441c-4d96-8d2a-6a73c887432d None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:31.615380 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-aed37bc1-441c-4d96-8d2a-6a73c887432d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:31.657200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe2139-b240-405d-93bf-48b09bcac7df None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:31.698040 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6ff30-153c-4a18-8ef8-75279759278d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:31.698846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6ff30-153c-4a18-8ef8-75279759278d None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:31.699156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6ff30-153c-4a18-8ef8-75279759278d None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:31.699429 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6ff30-153c-4a18-8ef8-75279759278d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:31.702445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 418/829] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:31.703883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed37bc1-441c-4d96-8d2a-6a73c887432d None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:31.707716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6ff30-153c-4a18-8ef8-75279759278d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:31.708662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6ff30-153c-4a18-8ef8-75279759278d None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:31.723707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-96b6ff30-153c-4a18-8ef8-75279759278d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:31.723707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:31.723707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:31.723707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:31.723707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:31.723707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:31.723707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:31.723707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:31.723707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:31.723707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:31.723707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:31.723707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:31.723707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:31.723707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:31.723707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:31.724638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:31.724638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:31.724638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:31.724638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:31.724638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:31.724638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:31.724638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:31.724638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:31.724638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:31.724638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:31.724638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:31.724638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:31.724638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:31.724638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:31.724638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:31.724638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:31.724638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:31.725634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:31.725634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:31.725634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:31.725634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:31.725634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:31.725634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:31.725634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:31.725634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:31.729885 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 412/830] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:31.731130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6ff30-153c-4a18-8ef8-75279759278d None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:31.747637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:31.786814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:31.811145 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed37bc1-441c-4d96-8d2a-6a73c887432d None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:31.843965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6ff30-153c-4a18-8ef8-75279759278d None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:31.847686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fdc3a-e58a-413a-b54c-7571ee15beae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:31.848510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fdc3a-e58a-413a-b54c-7571ee15beae None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:31.848887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fdc3a-e58a-413a-b54c-7571ee15beae None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:31.849255 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fdc3a-e58a-413a-b54c-7571ee15beae None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:31.850070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fdc3a-e58a-413a-b54c-7571ee15beae None admin] RBAC: Adding query filter params (type=network) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:31.850550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fdc3a-e58a-413a-b54c-7571ee15beae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:31.852070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fdc3a-e58a-413a-b54c-7571ee15beae None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:31.858017 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 419/831] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jul 21 05:31:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:31.858901 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fdc3a-e58a-413a-b54c-7571ee15beae None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:31.882860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79879cc3-d78b-41e0-8525-46966ffa7a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:31.884092 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79879cc3-d78b-41e0-8525-46966ffa7a65 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:31.884923 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79879cc3-d78b-41e0-8525-46966ffa7a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:31.885514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79879cc3-d78b-41e0-8525-46966ffa7a65 None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:31.886564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79879cc3-d78b-41e0-8525-46966ffa7a65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:31.887291 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79879cc3-d78b-41e0-8525-46966ffa7a65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:31.889583 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79879cc3-d78b-41e0-8525-46966ffa7a65 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:31.906301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 413/832] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:31.907501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79879cc3-d78b-41e0-8525-46966ffa7a65 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:31.971455 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fdc3a-e58a-413a-b54c-7571ee15beae None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:31.973409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:32.014842 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa33cd-3d69-479c-b12a-27495b8dcfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:32.016487 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa33cd-3d69-479c-b12a-27495b8dcfca None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:32.016963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa33cd-3d69-479c-b12a-27495b8dcfca None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:32.017388 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa33cd-3d69-479c-b12a-27495b8dcfca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:32.033156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79879cc3-d78b-41e0-8525-46966ffa7a65 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:32.033815 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa33cd-3d69-479c-b12a-27495b8dcfca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:32.044011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-42fa33cd-3d69-479c-b12a-27495b8dcfca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:31:32.044011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:32.044011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:32.044011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:32.044011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:32.044011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:32.044011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:32.044011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:32.044011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:32.044011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:32.044011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:32.044011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:32.044011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:32.044011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:32.044011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:32.044011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:32.045024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:32.045024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:32.045024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:32.045024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:32.045024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:32.045024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:32.045024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:32.045024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:32.045024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:32.045024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:32.045024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:32.045024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:32.045024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:32.045024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:32.045024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:32.045024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:32.045886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:32.045886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:32.045886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:32.045886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:32.045886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:32.045886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:32.045886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:32.045886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:32.045886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:32.045886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:32.045886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:31:32.045886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:32.047177 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:32.048612 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 420/833] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:31:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:32.049140 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa33cd-3d69-479c-b12a-27495b8dcfca None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:32.074928 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11039e56-af32-40af-9420-c791c0245cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:32.075624 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11039e56-af32-40af-9420-c791c0245cfd None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:32.075901 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11039e56-af32-40af-9420-c791c0245cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:32.076170 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11039e56-af32-40af-9420-c791c0245cfd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:32.076922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11039e56-af32-40af-9420-c791c0245cfd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:32.078276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11039e56-af32-40af-9420-c791c0245cfd None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:32.111534 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 414/834] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jul 21 05:31:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:31:32.112672 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11039e56-af32-40af-9420-c791c0245cfd None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:32.185112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa33cd-3d69-479c-b12a-27495b8dcfca None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:32.270712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e64ce-be57-4d8f-8aab-7077ea68644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:32.271788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e64ce-be57-4d8f-8aab-7077ea68644c None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:32.272373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e64ce-be57-4d8f-8aab-7077ea68644c None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:32.272851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e64ce-be57-4d8f-8aab-7077ea68644c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:32.274033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e64ce-be57-4d8f-8aab-7077ea68644c None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=928d014648cd41d7b58e7e01845eb748) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:32.276096 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e64ce-be57-4d8f-8aab-7077ea68644c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:32.277880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535e64ce-be57-4d8f-8aab-7077ea68644c None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:32.292350 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 421/835] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:32 2022] GET /identity/v3/role_assignments?user.id=928d014648cd41d7b58e7e01845eb748&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:32.311193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11039e56-af32-40af-9420-c791c0245cfd None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:32.362500 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ac46d-9f46-42f8-b74d-61fa12753827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:32.363257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ac46d-9f46-42f8-b74d-61fa12753827 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:32.363537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ac46d-9f46-42f8-b74d-61fa12753827 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:32.363861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ac46d-9f46-42f8-b74d-61fa12753827 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:32.364645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ac46d-9f46-42f8-b74d-61fa12753827 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:32.364994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ac46d-9f46-42f8-b74d-61fa12753827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:32.366469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ac46d-9f46-42f8-b74d-61fa12753827 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:32.378833 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 415/836] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:31:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:32.384575 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e64ce-be57-4d8f-8aab-7077ea68644c None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:32.423396 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:32.510727 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e64ce-be57-4d8f-8aab-7077ea68644c None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:32.551460 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93866f9-99e4-4a81-8166-90fb5cc91610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:32.552192 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93866f9-99e4-4a81-8166-90fb5cc91610 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:32.552470 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93866f9-99e4-4a81-8166-90fb5cc91610 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:32.552743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93866f9-99e4-4a81-8166-90fb5cc91610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:32.564306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93866f9-99e4-4a81-8166-90fb5cc91610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:32.565771 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93866f9-99e4-4a81-8166-90fb5cc91610 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:32.583619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-c93866f9-99e4-4a81-8166-90fb5cc91610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:32.583619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:32.583619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:32.583619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:32.583619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:32.583619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:32.583619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:32.583619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:32.583619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:32.583619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:32.583619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:32.583619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:32.583619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:32.583619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:32.583619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:32.584594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:32.584594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:32.584594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:32.584594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:32.584594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:32.584594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:32.584594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:32.584594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:32.584594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:32.584594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:32.584594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:32.584594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:32.584594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:32.584594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:32.584594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:32.584594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:32.584594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:32.585709 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:32.585709 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:32.585709 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:32.585709 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:32.585709 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 422/837] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:32.592114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ac46d-9f46-42f8-b74d-61fa12753827 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:32.621761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:32.709629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ac46d-9f46-42f8-b74d-61fa12753827 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:32.745103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd840a1-3cb1-4b47-927a-58958981a025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:32.745776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd840a1-3cb1-4b47-927a-58958981a025 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:32.746058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd840a1-3cb1-4b47-927a-58958981a025 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:32.746325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd840a1-3cb1-4b47-927a-58958981a025 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:32.747072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd840a1-3cb1-4b47-927a-58958981a025 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:32.747381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd840a1-3cb1-4b47-927a-58958981a025 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:32.748761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd840a1-3cb1-4b47-927a-58958981a025 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:32.761117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 416/838] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:32.766468 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93866f9-99e4-4a81-8166-90fb5cc91610 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:32.838867 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a18ce-7599-4b72-aa2f-73764ec932c5 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:32.839252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a18ce-7599-4b72-aa2f-73764ec932c5 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:32.839887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a18ce-7599-4b72-aa2f-73764ec932c5 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:32.841534 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 417/839] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:31:32.846591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb85c6-b4ab-4a5c-9d9e-662804c07f12 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:32.846939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb85c6-b4ab-4a5c-9d9e-662804c07f12 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:32.847328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb85c6-b4ab-4a5c-9d9e-662804c07f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:32.850103 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:32.865434 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-86fb85c6-b4ab-4a5c-9d9e-662804c07f12 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:32.866913 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-86fb85c6-b4ab-4a5c-9d9e-662804c07f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:32.876492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93866f9-99e4-4a81-8166-90fb5cc91610 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:32.915796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9e1b6-1716-4ac7-bfb5-df5a579cad30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:32.917214 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9e1b6-1716-4ac7-bfb5-df5a579cad30 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:32.917793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9e1b6-1716-4ac7-bfb5-df5a579cad30 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:32.918216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9e1b6-1716-4ac7-bfb5-df5a579cad30 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:32.928926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9e1b6-1716-4ac7-bfb5-df5a579cad30 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:32.931412 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9e1b6-1716-4ac7-bfb5-df5a579cad30 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:32.939369 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 418/840] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:32.946949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-c2b9e1b6-1716-4ac7-bfb5-df5a579cad30 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 21 05:31:32.946949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:32.946949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:32.946949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:32.946949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:32.946949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:32.946949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:32.946949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:32.946949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:32.946949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:32.946949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:32.946949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:32.946949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:32.946949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:32.946949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:32.946949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:32.948114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:32.949324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:32.949324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:32.949324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:32.949324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:32.949324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:32.949324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:32.949324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:32.949324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:32.949324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 21 05:31:32.949324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:32.951672 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 423/841] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:31:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:32.957097 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb85c6-b4ab-4a5c-9d9e-662804c07f12 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:32.964005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9e1b6-1716-4ac7-bfb5-df5a579cad30 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:32.991427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:33.065286 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9e1b6-1716-4ac7-bfb5-df5a579cad30 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:33.076902 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb85c6-b4ab-4a5c-9d9e-662804c07f12 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:33.100054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928d8f5-1516-4e0e-9ff0-4dade7f409f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:33.100870 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928d8f5-1516-4e0e-9ff0-4dade7f409f3 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:33.101159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928d8f5-1516-4e0e-9ff0-4dade7f409f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:33.101419 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928d8f5-1516-4e0e-9ff0-4dade7f409f3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:33.102077 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928d8f5-1516-4e0e-9ff0-4dade7f409f3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:33.103526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928d8f5-1516-4e0e-9ff0-4dade7f409f3 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:33.105835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:33.112232 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e86849-23f4-4490-b2eb-2cbe54464286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:33.112912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e86849-23f4-4490-b2eb-2cbe54464286 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:33.113188 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e86849-23f4-4490-b2eb-2cbe54464286 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:33.113457 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e86849-23f4-4490-b2eb-2cbe54464286 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:33.114296 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e86849-23f4-4490-b2eb-2cbe54464286 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:33.114600 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e86849-23f4-4490-b2eb-2cbe54464286 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:33.116097 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e86849-23f4-4490-b2eb-2cbe54464286 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:33.117375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-b928d8f5-1516-4e0e-9ff0-4dade7f409f3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 05:31:33.117375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:33.117375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:33.117375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:33.117375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:33.117375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:33.117375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:33.117375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:33.117375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 05:31:33.117375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:33.117375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:33.117375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:33.117375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:33.117375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:33.117375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:33.117375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:33.117375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:33.118509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:33.118509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:33.118509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:33.118509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:33.118509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:33.118509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:33.118509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:33.118509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:33.118509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:33.118509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:33.118509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:33.118509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:33.118509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:33.118509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:33.118509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:33.118509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:33.119424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:33.119424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:33.119424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:33.119424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:33.119424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:33.119424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:33.119424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:33.119424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:33.119424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:33.119424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:33.119424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:33.119424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:33.119424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 05:31:33.119424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:33.120339 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 424/842] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:31:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:33.121598 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae96741-abe5-437f-abb4-025acc5d25ee None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:33.121879 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae96741-abe5-437f-abb4-025acc5d25ee None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:33.122151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae96741-abe5-437f-abb4-025acc5d25ee None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:33.127368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 425/843] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:33.128113 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae96741-abe5-437f-abb4-025acc5d25ee None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:33.139972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 419/844] 10.222.0.44 () {56 vars in 1135 bytes} [Thu Jul 21 05:31:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:33.141575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5afcf8-b2ab-4ae9-ad7a-2e8474eb62a1 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:33.141865 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5afcf8-b2ab-4ae9-ad7a-2e8474eb62a1 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:33.142144 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5afcf8-b2ab-4ae9-ad7a-2e8474eb62a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:33.159628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-8a5afcf8-b2ab-4ae9-ad7a-2e8474eb62a1 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:33.160806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8a5afcf8-b2ab-4ae9-ad7a-2e8474eb62a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:33.230649 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 420/845] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:33.231539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5afcf8-b2ab-4ae9-ad7a-2e8474eb62a1 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:33.243196 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae96741-abe5-437f-abb4-025acc5d25ee None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:33.243891 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:33.281505 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdb464-4d48-41bb-b43f-23986cfa4344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:33.282178 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdb464-4d48-41bb-b43f-23986cfa4344 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:33.282462 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdb464-4d48-41bb-b43f-23986cfa4344 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:33.282737 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdb464-4d48-41bb-b43f-23986cfa4344 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:33.283476 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdb464-4d48-41bb-b43f-23986cfa4344 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:33.283783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdb464-4d48-41bb-b43f-23986cfa4344 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:33.285196 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdb464-4d48-41bb-b43f-23986cfa4344 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:33.294642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 426/846] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:31:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:33.294642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdb464-4d48-41bb-b43f-23986cfa4344 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:33.344009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5afcf8-b2ab-4ae9-ad7a-2e8474eb62a1 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:33.354502 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:33.381102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21711ce7-c1b7-4f93-a1e1-0825b442c0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:33.381979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21711ce7-c1b7-4f93-a1e1-0825b442c0de None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:33.382261 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21711ce7-c1b7-4f93-a1e1-0825b442c0de None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:33.382524 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21711ce7-c1b7-4f93-a1e1-0825b442c0de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:33.398498 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdb464-4d48-41bb-b43f-23986cfa4344 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:33.434890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1cd0e-76a1-46de-8c22-7222c57b6b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:33.435723 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1cd0e-76a1-46de-8c22-7222c57b6b4a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:33.436002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1cd0e-76a1-46de-8c22-7222c57b6b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:33.436283 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1cd0e-76a1-46de-8c22-7222c57b6b4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:33.447223 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1cd0e-76a1-46de-8c22-7222c57b6b4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:33.454720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-c6f1cd0e-76a1-46de-8c22-7222c57b6b4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:33.454720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:33.454720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:33.454720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:33.454720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:33.454720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:33.454720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:33.454720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:33.454720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:33.454720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:33.454720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:33.454720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:33.454720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:33.454720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:33.454720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:33.454720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:33.455905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:33.455905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:33.455905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:33.455905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:33.455905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:33.455905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:33.455905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:33.455905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:33.455905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:33.455905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:33.455905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:33.455905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:33.455905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:33.455905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:33.455905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:33.455905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:33.457053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:33.457053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:33.457053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:33.457053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:33.457053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:33.457053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:33.457053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:33.457053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:33.457053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:33.457053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:33.457053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:31:33.457053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:33.457053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 427/847] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:31:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:33.461535 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1cd0e-76a1-46de-8c22-7222c57b6b4a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:33.462488 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21711ce7-c1b7-4f93-a1e1-0825b442c0de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:33.464146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21711ce7-c1b7-4f93-a1e1-0825b442c0de None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:33.472957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:33.563158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1cd0e-76a1-46de-8c22-7222c57b6b4a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:33.577605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 421/848] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:33 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 05:31:33.578489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21711ce7-c1b7-4f93-a1e1-0825b442c0de None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:33.594624 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:33.610113 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bc232-d5a5-43af-9362-26d5b6290f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:33.610692 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bc232-d5a5-43af-9362-26d5b6290f7f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:33.610973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bc232-d5a5-43af-9362-26d5b6290f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:33.611272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bc232-d5a5-43af-9362-26d5b6290f7f None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:33.611993 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39bc232-d5a5-43af-9362-26d5b6290f7f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:33.612278 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39bc232-d5a5-43af-9362-26d5b6290f7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:33.613657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39bc232-d5a5-43af-9362-26d5b6290f7f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:33.623058 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 428/849] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:31:33 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:33.623058 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bc232-d5a5-43af-9362-26d5b6290f7f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:33.710340 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21711ce7-c1b7-4f93-a1e1-0825b442c0de None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:33.720021 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:33.752774 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba6108-e973-45a0-9733-ec182c89d7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:33.754648 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba6108-e973-45a0-9733-ec182c89d7cb None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:33.755213 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba6108-e973-45a0-9733-ec182c89d7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:33.755719 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba6108-e973-45a0-9733-ec182c89d7cb None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:33.757263 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba6108-e973-45a0-9733-ec182c89d7cb None admin] RBAC: Adding query filter params (name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:33.758163 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba6108-e973-45a0-9733-ec182c89d7cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:33.760520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba6108-e973-45a0-9733-ec182c89d7cb None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:33.773953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 422/850] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:31:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:33.779276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba6108-e973-45a0-9733-ec182c89d7cb None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:33.860258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bc232-d5a5-43af-9362-26d5b6290f7f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:33.895308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba6108-e973-45a0-9733-ec182c89d7cb None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:33.901490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea4f37-a1d9-4a7c-a8c5-e0607f6f2ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:33.902318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea4f37-a1d9-4a7c-a8c5-e0607f6f2ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:33.902600 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea4f37-a1d9-4a7c-a8c5-e0607f6f2ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:33.902875 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea4f37-a1d9-4a7c-a8c5-e0607f6f2ac2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:33.914072 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea4f37-a1d9-4a7c-a8c5-e0607f6f2ac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:33.915846 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea4f37-a1d9-4a7c-a8c5-e0607f6f2ac2 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:33.933774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-8bea4f37-a1d9-4a7c-a8c5-e0607f6f2ac2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:33.933774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:33.933774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:33.933774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:33.933774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:33.933774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:33.933774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:33.933774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:33.933774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:33.933774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:33.933774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:33.933774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:33.933774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:33.933774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:33.933774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:33.934711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:33.934711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:33.934711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:33.934711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:33.934711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:33.934711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:33.934711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:33.934711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:33.934711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:33.934711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:33.934711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:33.934711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:33.934711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:33.934711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:33.934711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:33.934711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:33.934711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:33.935825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:33.935825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:33.935825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:33.935825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:33.935825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 429/851] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:33.940449 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea4f37-a1d9-4a7c-a8c5-e0607f6f2ac2 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:33.945679 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72975a-b5b4-4d3f-b4b9-e4b075b82cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:33.946548 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72975a-b5b4-4d3f-b4b9-e4b075b82cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:33.946828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72975a-b5b4-4d3f-b4b9-e4b075b82cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:33.948002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72975a-b5b4-4d3f-b4b9-e4b075b82cd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:33.961502 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72975a-b5b4-4d3f-b4b9-e4b075b82cd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:33.962977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72975a-b5b4-4d3f-b4b9-e4b075b82cd6 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:33.984450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-fe72975a-b5b4-4d3f-b4b9-e4b075b82cd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:33.984450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:33.984450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:33.984450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:33.984450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:33.984450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:33.984450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:33.984450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:33.984450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:33.984450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:33.984450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:33.984450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:33.984450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:33.984450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:33.984450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:33.985419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:33.985419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:33.985419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:33.985419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:33.985419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:33.985419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:33.985419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:33.985419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:33.985419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:33.985419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:33.985419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:33.985419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:33.985419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:33.985419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:33.985419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:33.985419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:33.985419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:33.986301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:33.986301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:33.986301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:33.986301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:33.990110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:33.993293 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 423/852] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:33.994630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72975a-b5b4-4d3f-b4b9-e4b075b82cd6 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:34.019781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:34.071548 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea4f37-a1d9-4a7c-a8c5-e0607f6f2ac2 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:34.108485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72975a-b5b4-4d3f-b4b9-e4b075b82cd6 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:34.124006 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39dcd9-ebe7-4c05-9888-b4308c0f0731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:34.125450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39dcd9-ebe7-4c05-9888-b4308c0f0731 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:34.125970 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39dcd9-ebe7-4c05-9888-b4308c0f0731 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:34.126403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39dcd9-ebe7-4c05-9888-b4308c0f0731 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:34.127565 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39dcd9-ebe7-4c05-9888-b4308c0f0731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:34.128400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39dcd9-ebe7-4c05-9888-b4308c0f0731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:34.130003 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39dcd9-ebe7-4c05-9888-b4308c0f0731 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:34.145737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63787f57-e60a-4e23-ae59-57dad5794928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:34.146408 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63787f57-e60a-4e23-ae59-57dad5794928 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:34.146690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63787f57-e60a-4e23-ae59-57dad5794928 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:34.146960 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63787f57-e60a-4e23-ae59-57dad5794928 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:34.147919 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63787f57-e60a-4e23-ae59-57dad5794928 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:34.148232 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63787f57-e60a-4e23-ae59-57dad5794928 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:34.149585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 430/853] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:34.149716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63787f57-e60a-4e23-ae59-57dad5794928 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:34.150854 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39dcd9-ebe7-4c05-9888-b4308c0f0731 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:34.167113 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 424/854] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:34.170575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63787f57-e60a-4e23-ae59-57dad5794928 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:34.270746 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:34.274738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39dcd9-ebe7-4c05-9888-b4308c0f0731 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:34.275822 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:34.300802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63787f57-e60a-4e23-ae59-57dad5794928 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:34.312934 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89697e66-505a-4415-b310-8a79342d40c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:34.314118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89697e66-505a-4415-b310-8a79342d40c5 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:34.314298 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89697e66-505a-4415-b310-8a79342d40c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:34.314648 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89697e66-505a-4415-b310-8a79342d40c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:34.328835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89697e66-505a-4415-b310-8a79342d40c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:34.330445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89697e66-505a-4415-b310-8a79342d40c5 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:34.343489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad056972-2cd2-4266-a674-e42fff4c2fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:34.344525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad056972-2cd2-4266-a674-e42fff4c2fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:34.345406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-89697e66-505a-4415-b310-8a79342d40c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:34.345406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:34.345406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:34.345406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:34.345406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:34.345406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:34.345406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:34.345406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:34.345406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:34.345406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:34.345406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:34.345406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:34.345406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:34.345406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:34.345406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:34.346504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:34.346504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:34.346504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:34.346504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:34.346504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:34.346504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:34.346504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:34.346504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:34.346504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:34.346504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:34.346504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:34.346504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:34.346504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:34.346504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:34.346504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:34.346504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:34.346504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:34.347689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:34.347689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:34.347689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:34.347689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:34.347689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:34.347689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:34.347689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:34.347689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:34.348395 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad056972-2cd2-4266-a674-e42fff4c2fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:34.348833 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad056972-2cd2-4266-a674-e42fff4c2fa8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:34.354299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 431/855] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:34.355629 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89697e66-505a-4415-b310-8a79342d40c5 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:34.357071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad056972-2cd2-4266-a674-e42fff4c2fa8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:34.359003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad056972-2cd2-4266-a674-e42fff4c2fa8 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:34.379268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-ad056972-2cd2-4266-a674-e42fff4c2fa8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 05:31:34.379268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:34.379268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:34.379268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:34.379268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:34.379268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:34.379268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:34.379268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:34.379268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:34.379268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:34.379268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:34.379268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:34.379268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:34.379268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:34.379268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:34.379268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:34.380372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:34.381329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:34.381329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:34.381329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:34.381329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:34.381329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:34.381329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:34.381329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:34.381329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:34.381329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 05:31:34.381329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:34.388609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 425/856] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jul 21 05:31:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:34.389298 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad056972-2cd2-4266-a674-e42fff4c2fa8 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:34.478639 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89697e66-505a-4415-b310-8a79342d40c5 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:34.516019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c65c62-9b35-4c55-8626-ddeceeb743c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:34.516945 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c65c62-9b35-4c55-8626-ddeceeb743c1 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:34.517232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c65c62-9b35-4c55-8626-ddeceeb743c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:34.517506 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c65c62-9b35-4c55-8626-ddeceeb743c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:34.518277 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad056972-2cd2-4266-a674-e42fff4c2fa8 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:34.518376 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c65c62-9b35-4c55-8626-ddeceeb743c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:34.518577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c65c62-9b35-4c55-8626-ddeceeb743c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:34.520039 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c65c62-9b35-4c55-8626-ddeceeb743c1 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:34.532970 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 432/857] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:34.534291 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27e434-14b5-47fc-b774-f3d16aa08cca None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:34.534573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27e434-14b5-47fc-b774-f3d16aa08cca None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:34.534844 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27e434-14b5-47fc-b774-f3d16aa08cca None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:34.540654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 433/858] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:34.541413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27e434-14b5-47fc-b774-f3d16aa08cca None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:34.560355 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf53d1-e753-4ebb-9097-52bcb07016ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:34.560920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf53d1-e753-4ebb-9097-52bcb07016ec None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:34.561218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf53d1-e753-4ebb-9097-52bcb07016ec None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:34.561419 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:34.561522 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf53d1-e753-4ebb-9097-52bcb07016ec None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:34.562345 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf53d1-e753-4ebb-9097-52bcb07016ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:34.562658 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf53d1-e753-4ebb-9097-52bcb07016ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:34.564192 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf53d1-e753-4ebb-9097-52bcb07016ec None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:34.574854 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:34.584407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 426/859] 10.222.0.44 () {56 vars in 1149 bytes} [Thu Jul 21 05:31:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:34.588349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988f058-f6b9-48b5-9c89-77a4ca36fa24 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:34.588638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988f058-f6b9-48b5-9c89-77a4ca36fa24 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:34.588902 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988f058-f6b9-48b5-9c89-77a4ca36fa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:34.606788 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-d988f058-f6b9-48b5-9c89-77a4ca36fa24 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:34.608037 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d988f058-f6b9-48b5-9c89-77a4ca36fa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:34.653655 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27e434-14b5-47fc-b774-f3d16aa08cca None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:34.680456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 427/860] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:34.684278 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988f058-f6b9-48b5-9c89-77a4ca36fa24 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:34.689608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf607ea-923e-4c59-a0b3-f8675a3d1da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:34.690276 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf607ea-923e-4c59-a0b3-f8675a3d1da4 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:34.690559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf607ea-923e-4c59-a0b3-f8675a3d1da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:34.690832 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf607ea-923e-4c59-a0b3-f8675a3d1da4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:34.691698 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf607ea-923e-4c59-a0b3-f8675a3d1da4 None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=61b61188b5b94a51ba1ae1a88b78b89f) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:34.692009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf607ea-923e-4c59-a0b3-f8675a3d1da4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:34.693447 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf607ea-923e-4c59-a0b3-f8675a3d1da4 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:34.703054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 434/861] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:34 2022] GET /identity/v3/role_assignments?user.id=61b61188b5b94a51ba1ae1a88b78b89f&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:34.703054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf607ea-923e-4c59-a0b3-f8675a3d1da4 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:34.813092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:34.836886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf607ea-923e-4c59-a0b3-f8675a3d1da4 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:34.837717 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988f058-f6b9-48b5-9c89-77a4ca36fa24 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:34.876371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:34.885094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11fb4b-bcba-4c80-9c31-6945db1aa90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:34.885942 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11fb4b-bcba-4c80-9c31-6945db1aa90f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:34.886213 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11fb4b-bcba-4c80-9c31-6945db1aa90f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:34.886485 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11fb4b-bcba-4c80-9c31-6945db1aa90f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:34.887185 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11fb4b-bcba-4c80-9c31-6945db1aa90f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:34.888568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11fb4b-bcba-4c80-9c31-6945db1aa90f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:34.889126 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee1178-a00b-4e27-808f-2cb4cae28b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:34.889761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee1178-a00b-4e27-808f-2cb4cae28b1a None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:34.890046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee1178-a00b-4e27-808f-2cb4cae28b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:34.890444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee1178-a00b-4e27-808f-2cb4cae28b1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:34.902840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-ef11fb4b-bcba-4c80-9c31-6945db1aa90f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 05:31:34.902840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:34.902840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:34.902840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:34.902840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:34.902840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:34.902840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:34.902840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:34.902840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 05:31:34.902840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:34.902840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:34.902840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:34.902840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:34.902840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:34.902840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:34.902840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:34.903974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:34.903974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:34.903974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:34.903974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:34.903974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:34.903974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:34.903974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:34.903974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:34.903974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:34.903974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:34.903974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:34.903974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:34.903974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:34.903974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:34.903974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:34.903974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:34.904877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:34.904877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:34.904877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:34.904877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:34.904877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:34.904877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:34.904877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:34.904877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:34.904877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:34.904877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:34.904877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:34.904877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:34.904877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:34.904877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 05:31:34.904877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:34.906343 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 435/862] 10.222.0.44 () {56 vars in 1132 bytes} [Thu Jul 21 05:31:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:34.910297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11fb4b-bcba-4c80-9c31-6945db1aa90f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:34.969904 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee1178-a00b-4e27-808f-2cb4cae28b1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:34.971575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee1178-a00b-4e27-808f-2cb4cae28b1a None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:35.016683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11fb4b-bcba-4c80-9c31-6945db1aa90f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:35.052804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca71ae-c64a-432e-853e-957f63216809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:35.053574 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca71ae-c64a-432e-853e-957f63216809 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:35.053864 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca71ae-c64a-432e-853e-957f63216809 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:35.054134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca71ae-c64a-432e-853e-957f63216809 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:35.054934 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca71ae-c64a-432e-853e-957f63216809 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:35.055300 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca71ae-c64a-432e-853e-957f63216809 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:35.056854 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ca71ae-c64a-432e-853e-957f63216809 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:35.066745 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 436/863] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jul 21 05:31:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:35.067742 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca71ae-c64a-432e-853e-957f63216809 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:35.081012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:35.097884 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 428/864] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:34 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 05:31:35.098928 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee1178-a00b-4e27-808f-2cb4cae28b1a None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:35.133224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:35.204040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca71ae-c64a-432e-853e-957f63216809 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:35.216055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee1178-a00b-4e27-808f-2cb4cae28b1a None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:35.238347 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b40ce8-4f99-469b-9c08-e253cf31778f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:35.239041 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b40ce8-4f99-469b-9c08-e253cf31778f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:35.239344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b40ce8-4f99-469b-9c08-e253cf31778f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:35.239618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b40ce8-4f99-469b-9c08-e253cf31778f None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:35.240368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b40ce8-4f99-469b-9c08-e253cf31778f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:35.240674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b40ce8-4f99-469b-9c08-e253cf31778f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:35.242237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b40ce8-4f99-469b-9c08-e253cf31778f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:35.255364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 437/865] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jul 21 05:31:35 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:35.262469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feae468-06f7-414f-8216-b2c67f051a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:35.263741 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feae468-06f7-414f-8216-b2c67f051a05 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:35.264387 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feae468-06f7-414f-8216-b2c67f051a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:35.264899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feae468-06f7-414f-8216-b2c67f051a05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:35.277235 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feae468-06f7-414f-8216-b2c67f051a05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:35.278755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feae468-06f7-414f-8216-b2c67f051a05 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:35.298517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-1feae468-06f7-414f-8216-b2c67f051a05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:35.298517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:35.298517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:35.298517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:35.298517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:35.298517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:35.298517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:35.298517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:35.298517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:35.298517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:35.298517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:35.298517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:35.298517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:35.298517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:35.298517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:35.299532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:35.299532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:35.299532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:35.299532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:35.299532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:35.299532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:35.299532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:35.299532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:35.299532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:35.299532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:35.299532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:35.299532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:35.299532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:35.299532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:35.299532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:35.299532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:35.299532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:35.300893 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:35.300893 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:35.300893 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:35.300893 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:35.300893 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 429/866] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:35.306156 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b40ce8-4f99-469b-9c08-e253cf31778f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:35.389978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:35.417052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b40ce8-4f99-469b-9c08-e253cf31778f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:35.453651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbac61c-bb34-48b8-905a-089610bad2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:35.454380 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbac61c-bb34-48b8-905a-089610bad2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:35.454601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbac61c-bb34-48b8-905a-089610bad2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:35.454873 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbac61c-bb34-48b8-905a-089610bad2c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:35.455627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbac61c-bb34-48b8-905a-089610bad2c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:35.455928 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbac61c-bb34-48b8-905a-089610bad2c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:35.457298 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbac61c-bb34-48b8-905a-089610bad2c6 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:35.470916 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 438/867] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:35.476526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feae468-06f7-414f-8216-b2c67f051a05 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:35.559185 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:35.578936 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feae468-06f7-414f-8216-b2c67f051a05 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:35.613454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45263b2-c950-4674-ac09-29ef5c12a19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:35.614263 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45263b2-c950-4674-ac09-29ef5c12a19b None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:35.614540 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45263b2-c950-4674-ac09-29ef5c12a19b None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:35.614808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45263b2-c950-4674-ac09-29ef5c12a19b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:35.621918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45263b2-c950-4674-ac09-29ef5c12a19b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:35.623406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45263b2-c950-4674-ac09-29ef5c12a19b None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:35.637155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-c45263b2-c950-4674-ac09-29ef5c12a19b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:35.637155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:35.637155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:35.637155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:35.637155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:35.637155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:35.637155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:35.637155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:35.637155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:35.637155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:35.637155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:35.637155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:35.637155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:35.637155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:35.637155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:35.638061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:35.638061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:35.638061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:35.638061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:35.638061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:35.638061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:35.638061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:35.638061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:35.638061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:35.638061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:35.638061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:35.638061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:35.638061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:35.638061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:35.638061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:35.638061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:35.638061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:35.638944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:35.638944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:35.638944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:35.638944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:35.638944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:35.638944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:35.638944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:35.638944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:35.638944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 430/868] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:35.648172 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbac61c-bb34-48b8-905a-089610bad2c6 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:35.751940 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbac61c-bb34-48b8-905a-089610bad2c6 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:35.780226 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461274a0-1028-42bb-9f5f-0e0a436948b0 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:35.780583 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461274a0-1028-42bb-9f5f-0e0a436948b0 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:35.780779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461274a0-1028-42bb-9f5f-0e0a436948b0 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:35.782300 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 431/869] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:31:35.787290 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a06877-fee7-419c-80dd-06d1d0573c99 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:35.788191 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a06877-fee7-419c-80dd-06d1d0573c99 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:35.788688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a06877-fee7-419c-80dd-06d1d0573c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:35.794502 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0567db-f885-4cad-9120-5d7233816311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:35.795272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0567db-f885-4cad-9120-5d7233816311 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:35.795553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0567db-f885-4cad-9120-5d7233816311 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:35.795826 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0567db-f885-4cad-9120-5d7233816311 None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:35.797448 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0567db-f885-4cad-9120-5d7233816311 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:35.797766 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0567db-f885-4cad-9120-5d7233816311 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:35.799223 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0567db-f885-4cad-9120-5d7233816311 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:35.807350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-90a06877-fee7-419c-80dd-06d1d0573c99 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:35.807499 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:35.808899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-90a06877-fee7-419c-80dd-06d1d0573c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:35.813472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 439/870] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:35.820244 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0567db-f885-4cad-9120-5d7233816311 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:35.899053 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 432/871] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:35.920015 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a06877-fee7-419c-80dd-06d1d0573c99 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:35.934900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0567db-f885-4cad-9120-5d7233816311 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:35.978284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceaa985-ccb5-4d7c-b741-d6ac22f24cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:35.979360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaa985-ccb5-4d7c-b741-d6ac22f24cbc None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:35.979646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaa985-ccb5-4d7c-b741-d6ac22f24cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:35.979924 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaa985-ccb5-4d7c-b741-d6ac22f24cbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:35.981436 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceaa985-ccb5-4d7c-b741-d6ac22f24cbc None admin] RBAC: Adding query filter params (role.id=bb94781ab8054924a5cb21b561b12bbb, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=4d04998b636649d98103b1cd5f3a0f7c) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:35.981753 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceaa985-ccb5-4d7c-b741-d6ac22f24cbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:35.984156 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceaa985-ccb5-4d7c-b741-d6ac22f24cbc None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:35.994139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 440/872] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:35 2022] GET /identity/v3/role_assignments?user.id=4d04998b636649d98103b1cd5f3a0f7c&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=bb94781ab8054924a5cb21b561b12bbb => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:36.020520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:36.057318 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a06877-fee7-419c-80dd-06d1d0573c99 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:36.094665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e708789-3ef1-4d89-a788-4ad89f206769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:36.095383 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e708789-3ef1-4d89-a788-4ad89f206769 None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:36.095667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e708789-3ef1-4d89-a788-4ad89f206769 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:36.095939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e708789-3ef1-4d89-a788-4ad89f206769 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:36.096685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e708789-3ef1-4d89-a788-4ad89f206769 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:36.098103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e708789-3ef1-4d89-a788-4ad89f206769 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:36.122635 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 433/873] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jul 21 05:31:35 2022] POST /identity/v3/services => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:31:36.887555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa1dd2-cbc7-4fba-9679-b005c6be7042 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:36.888060 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa1dd2-cbc7-4fba-9679-b005c6be7042 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:36.888443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa1dd2-cbc7-4fba-9679-b005c6be7042 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:36.890207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 441/874] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:36.895686 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f3779-af49-4cca-83d9-6e3d9ebcef73 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:36.896141 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f3779-af49-4cca-83d9-6e3d9ebcef73 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:36.896323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f3779-af49-4cca-83d9-6e3d9ebcef73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:36.913616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-0d0f3779-af49-4cca-83d9-6e3d9ebcef73 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:36.914729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-0d0f3779-af49-4cca-83d9-6e3d9ebcef73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:36.991180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 434/875] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:36 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:37.008593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa1dd2-cbc7-4fba-9679-b005c6be7042 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:37.070921 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:37.112850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa1dd2-cbc7-4fba-9679-b005c6be7042 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:37.145872 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e5568-6f72-4e1d-aa00-53d56bd1ffe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:37.146686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e5568-6f72-4e1d-aa00-53d56bd1ffe5 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:37.146964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e5568-6f72-4e1d-aa00-53d56bd1ffe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:37.147332 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e5568-6f72-4e1d-aa00-53d56bd1ffe5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:37.147978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e5568-6f72-4e1d-aa00-53d56bd1ffe5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:37.149350 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e5568-6f72-4e1d-aa00-53d56bd1ffe5 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:37.161894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-209e5568-6f72-4e1d-aa00-53d56bd1ffe5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 05:31:37.161894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:37.161894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:37.161894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:37.161894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:37.161894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:37.161894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:37.161894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:37.161894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 05:31:37.161894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:37.161894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:37.161894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:37.161894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:37.161894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:37.161894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:37.161894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:37.161894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:37.163305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:37.163305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:37.163305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:37.163305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:37.163305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:37.163305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:37.163305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:37.163305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:37.163305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:37.163305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:37.163305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:37.163305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:37.163305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:37.163305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:37.163305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:37.163305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:37.164174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:37.164174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:37.164174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:37.164174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:37.164174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:37.164174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:37.164174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:37.164174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:37.164174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:37.164174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:37.164174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:37.164174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:37.164174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 05:31:37.164174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:37.164174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 442/876] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jul 21 05:31:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:37.169513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f3779-af49-4cca-83d9-6e3d9ebcef73 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:37.230435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:37.276969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f3779-af49-4cca-83d9-6e3d9ebcef73 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:37.316058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc3909-b115-44d0-8dc1-9c35cae1d726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:37.317157 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc3909-b115-44d0-8dc1-9c35cae1d726 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:37.317632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc3909-b115-44d0-8dc1-9c35cae1d726 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:37.318103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc3909-b115-44d0-8dc1-9c35cae1d726 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:37.319059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc3909-b115-44d0-8dc1-9c35cae1d726 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:37.319648 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc3909-b115-44d0-8dc1-9c35cae1d726 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:37.321224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc3909-b115-44d0-8dc1-9c35cae1d726 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:37.326766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 435/877] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jul 21 05:31:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:37.335329 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e5568-6f72-4e1d-aa00-53d56bd1ffe5 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:37.381317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bab6e3-1252-4f26-9516-ded1f0779ffc None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:37.381843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bab6e3-1252-4f26-9516-ded1f0779ffc None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:37.382214 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bab6e3-1252-4f26-9516-ded1f0779ffc None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:37.383942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 436/878] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:31:37.388826 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c121d-3f15-42a7-bbb2-47a96b467645 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:37.389169 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c121d-3f15-42a7-bbb2-47a96b467645 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:37.389511 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c121d-3f15-42a7-bbb2-47a96b467645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:37.406451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-986c121d-3f15-42a7-bbb2-47a96b467645 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:37.407810 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-986c121d-3f15-42a7-bbb2-47a96b467645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:37.442380 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e5568-6f72-4e1d-aa00-53d56bd1ffe5 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:37.465468 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:37.475211 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 437/879] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:37 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:37.478023 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5e19b-46bf-4e3c-852c-fe51053c23ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:37.478911 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5e19b-46bf-4e3c-852c-fe51053c23ea None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:37.479394 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5e19b-46bf-4e3c-852c-fe51053c23ea None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:37.479913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5e19b-46bf-4e3c-852c-fe51053c23ea None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:37.480791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5e19b-46bf-4e3c-852c-fe51053c23ea None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:37.481272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5e19b-46bf-4e3c-852c-fe51053c23ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:37.482831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5e19b-46bf-4e3c-852c-fe51053c23ea None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:37.491116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 443/880] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jul 21 05:31:37 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:37.498493 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c121d-3f15-42a7-bbb2-47a96b467645 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:37.554725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:37.608587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c121d-3f15-42a7-bbb2-47a96b467645 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:37.645759 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb87c6f-6bfa-47de-a8a8-1035cd1a8023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:37.646455 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb87c6f-6bfa-47de-a8a8-1035cd1a8023 None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:37.646731 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb87c6f-6bfa-47de-a8a8-1035cd1a8023 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:37.647051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb87c6f-6bfa-47de-a8a8-1035cd1a8023 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:37.647916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb87c6f-6bfa-47de-a8a8-1035cd1a8023 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:37.649307 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb87c6f-6bfa-47de-a8a8-1035cd1a8023 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:37.679501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 438/881] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jul 21 05:31:37 2022] POST /identity/v3/services => generated 269 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:31:38.149091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5cda7-8730-49dd-a562-3bdd57965388 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:38.149618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5cda7-8730-49dd-a562-3bdd57965388 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:38.149697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5cda7-8730-49dd-a562-3bdd57965388 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:38.151325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 444/882] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:38.156470 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09162569-1c42-4fa5-8758-da78ed30a3dc None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:38.157276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09162569-1c42-4fa5-8758-da78ed30a3dc None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:38.157755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09162569-1c42-4fa5-8758-da78ed30a3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:38.175357 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-09162569-1c42-4fa5-8758-da78ed30a3dc None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:38.176871 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-09162569-1c42-4fa5-8758-da78ed30a3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:38.236910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fea34-0bf8-400e-9fec-e2f0e82d5037 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:38.237254 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fea34-0bf8-400e-9fec-e2f0e82d5037 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:38.237476 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fea34-0bf8-400e-9fec-e2f0e82d5037 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:38.239143 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 445/883] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:31:38.244364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dabcf8-e105-4bc4-8cd5-26e9bba6ba17 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:38.244652 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dabcf8-e105-4bc4-8cd5-26e9bba6ba17 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:38.244935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dabcf8-e105-4bc4-8cd5-26e9bba6ba17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:38.251168 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 439/884] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:38.262113 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-26dabcf8-e105-4bc4-8cd5-26e9bba6ba17 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:38.263355 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-26dabcf8-e105-4bc4-8cd5-26e9bba6ba17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:38.274976 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09162569-1c42-4fa5-8758-da78ed30a3dc None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:38.314404 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:38.355189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 446/885] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:38.378331 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dabcf8-e105-4bc4-8cd5-26e9bba6ba17 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:38.385598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09162569-1c42-4fa5-8758-da78ed30a3dc None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:38.423600 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be960cca-9bc0-4e33-b6c9-89cce79fd589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:38.424595 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be960cca-9bc0-4e33-b6c9-89cce79fd589 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:38.425022 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be960cca-9bc0-4e33-b6c9-89cce79fd589 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:38.425391 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be960cca-9bc0-4e33-b6c9-89cce79fd589 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:38.436960 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be960cca-9bc0-4e33-b6c9-89cce79fd589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:38.445894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-be960cca-9bc0-4e33-b6c9-89cce79fd589 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 05:31:38.445894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:38.445894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:38.445894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:38.445894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:38.445894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:38.445894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:38.445894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:38.445894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:38.445894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:38.445894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:38.445894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:38.445894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:38.445894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:38.445894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:38.445894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:38.447060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:38.447060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:38.447060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:38.447060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:38.447060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:38.447060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:38.447060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:38.447060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:38.447060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:38.447060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:38.447060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:38.447060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:38.447060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:38.447060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:38.447060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:38.447060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:38.447948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:38.447948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:38.447948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:38.447948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:38.447948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:38.447948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:38.447948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:38.447948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:38.447948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:38.447948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:38.447948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 05:31:38.447948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:38.448599 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 440/886] 10.222.0.44 () {56 vars in 1120 bytes} [Thu Jul 21 05:31:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:38.454853 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be960cca-9bc0-4e33-b6c9-89cce79fd589 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:38.484980 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dabcf8-e105-4bc4-8cd5-26e9bba6ba17 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:38.506385 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:38.526024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d0c73-102f-4856-b550-8f3eb0bcac1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:38.527199 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d0c73-102f-4856-b550-8f3eb0bcac1d None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:38.527688 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d0c73-102f-4856-b550-8f3eb0bcac1d None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:38.528163 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d0c73-102f-4856-b550-8f3eb0bcac1d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:38.529045 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d0c73-102f-4856-b550-8f3eb0bcac1d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:38.530661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d0c73-102f-4856-b550-8f3eb0bcac1d None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:38.546592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-982d0c73-102f-4856-b550-8f3eb0bcac1d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 05:31:38.546592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:38.546592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:38.546592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:38.546592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:38.546592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:38.546592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:38.546592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:38.546592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 05:31:38.546592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:38.546592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:38.546592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:38.546592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:38.546592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:38.546592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:38.546592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:38.546592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:38.548537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:38.548537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:38.548537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:38.548537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:38.548537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:38.548537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:38.548537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:38.548537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:38.548537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:38.548537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:38.548537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:38.548537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:38.548537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:38.548537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:38.548537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:38.548537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:38.550162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:38.550162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:38.550162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:38.550162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:38.550162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:38.550162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:38.550162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:38.550162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:38.550162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:38.550162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:38.550162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:38.550162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:38.550162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 05:31:38.550162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:38.556880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:38.559057 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 447/887] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:31:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:38.560142 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d0c73-102f-4856-b550-8f3eb0bcac1d None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:38.564584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be960cca-9bc0-4e33-b6c9-89cce79fd589 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:38.601105 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08df984-2e99-4345-b321-a5581c878dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:38.602082 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08df984-2e99-4345-b321-a5581c878dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:38.602504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08df984-2e99-4345-b321-a5581c878dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:38.602915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08df984-2e99-4345-b321-a5581c878dc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:38.603823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08df984-2e99-4345-b321-a5581c878dc7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:38.604282 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08df984-2e99-4345-b321-a5581c878dc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:38.605902 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08df984-2e99-4345-b321-a5581c878dc7 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:38.619781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 441/888] 10.222.0.44 () {56 vars in 1115 bytes} [Thu Jul 21 05:31:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:38.620855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08df984-2e99-4345-b321-a5581c878dc7 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:38.670386 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d0c73-102f-4856-b550-8f3eb0bcac1d None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:38.712392 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e3905-dabc-4adf-9f06-af88e871f58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:38.713169 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e3905-dabc-4adf-9f06-af88e871f58f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:38.713552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e3905-dabc-4adf-9f06-af88e871f58f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:38.713930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e3905-dabc-4adf-9f06-af88e871f58f None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:38.714740 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e3905-dabc-4adf-9f06-af88e871f58f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:38.715185 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e3905-dabc-4adf-9f06-af88e871f58f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:38.716769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e3905-dabc-4adf-9f06-af88e871f58f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:38.727261 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 448/889] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:31:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:38.728229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e3905-dabc-4adf-9f06-af88e871f58f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:38.729074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08df984-2e99-4345-b321-a5581c878dc7 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:38.763760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11deeef9-fa10-4780-ab49-d757b3cb066a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:38.764850 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11deeef9-fa10-4780-ab49-d757b3cb066a None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:38.765274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11deeef9-fa10-4780-ab49-d757b3cb066a None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:38.765686 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11deeef9-fa10-4780-ab49-d757b3cb066a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:38.776935 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11deeef9-fa10-4780-ab49-d757b3cb066a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:38.778620 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11deeef9-fa10-4780-ab49-d757b3cb066a None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:38.778927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:38.797663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-11deeef9-fa10-4780-ab49-d757b3cb066a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:38.797663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:38.797663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:38.797663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:38.797663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:38.797663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:38.797663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:38.797663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:38.797663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:38.797663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:38.797663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:38.797663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:38.797663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:38.797663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:38.797663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:38.798691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:38.798691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:38.798691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:38.798691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:38.798691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:38.798691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:38.798691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:38.798691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:38.798691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:38.798691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:38.798691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:38.798691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:38.798691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:38.798691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:38.798691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:38.798691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:38.798691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:38.799730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:38.799730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:38.799730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:38.799730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:38.806414 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 442/890] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:38.807609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11deeef9-fa10-4780-ab49-d757b3cb066a None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:38.820572 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:38.844600 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e3905-dabc-4adf-9f06-af88e871f58f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:38.881513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27e0a7-6776-4fe4-983a-a1d370557900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:38.882327 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27e0a7-6776-4fe4-983a-a1d370557900 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:38.882678 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27e0a7-6776-4fe4-983a-a1d370557900 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:38.883053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27e0a7-6776-4fe4-983a-a1d370557900 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:38.883838 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27e0a7-6776-4fe4-983a-a1d370557900 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:38.884212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27e0a7-6776-4fe4-983a-a1d370557900 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:38.885821 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27e0a7-6776-4fe4-983a-a1d370557900 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:38.896332 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 449/891] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:31:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:38.897403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27e0a7-6776-4fe4-983a-a1d370557900 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:38.914253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11deeef9-fa10-4780-ab49-d757b3cb066a None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:38.947915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02425147-14ce-431f-a1a1-1ee66b8f03f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:38.948943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02425147-14ce-431f-a1a1-1ee66b8f03f2 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:38.949441 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02425147-14ce-431f-a1a1-1ee66b8f03f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:38.949917 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02425147-14ce-431f-a1a1-1ee66b8f03f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:38.950840 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02425147-14ce-431f-a1a1-1ee66b8f03f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:38.951368 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02425147-14ce-431f-a1a1-1ee66b8f03f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:38.952957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02425147-14ce-431f-a1a1-1ee66b8f03f2 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:38.969884 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 443/892] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:38.970993 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02425147-14ce-431f-a1a1-1ee66b8f03f2 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:39.013112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27e0a7-6776-4fe4-983a-a1d370557900 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:39.022424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:39.042963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:39.052710 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725294f9-a7a8-4241-9501-d056a44a2465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:39.054242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725294f9-a7a8-4241-9501-d056a44a2465 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:39.054590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725294f9-a7a8-4241-9501-d056a44a2465 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:39.054874 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725294f9-a7a8-4241-9501-d056a44a2465 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:39.055720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725294f9-a7a8-4241-9501-d056a44a2465 None admin] RBAC: Adding query filter params (interface=public, service_id=3c5d18ef5cb940d581a6928ba8ed9f72, region_id=RegionOne) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:39.056033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725294f9-a7a8-4241-9501-d056a44a2465 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:39.057555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725294f9-a7a8-4241-9501-d056a44a2465 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:39.070555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 450/893] 10.222.0.44 () {56 vars in 1251 bytes} [Thu Jul 21 05:31:38 2022] GET /identity/v3/endpoints?service_id=3c5d18ef5cb940d581a6928ba8ed9f72&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:39.071575 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725294f9-a7a8-4241-9501-d056a44a2465 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:39.079547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02425147-14ce-431f-a1a1-1ee66b8f03f2 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:39.116227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe81d6-d81d-4e12-8ba1-b43d85a2496c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:39.117521 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe81d6-d81d-4e12-8ba1-b43d85a2496c None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:39.118004 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe81d6-d81d-4e12-8ba1-b43d85a2496c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:39.118475 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe81d6-d81d-4e12-8ba1-b43d85a2496c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:39.126794 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe81d6-d81d-4e12-8ba1-b43d85a2496c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:39.128599 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe81d6-d81d-4e12-8ba1-b43d85a2496c None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:39.144427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-ddbe81d6-d81d-4e12-8ba1-b43d85a2496c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 05:31:39.144427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:39.144427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:39.144427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:39.144427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:39.144427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:39.144427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:39.144427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:39.144427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:39.144427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:39.144427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:39.144427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:39.144427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:39.144427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:39.144427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:39.144427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:39.145461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:39.146482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:39.146482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:39.146482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:39.146482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:39.146482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:39.146482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:39.146482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:39.146482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:39.146482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 05:31:39.146482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:39.153799 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 444/894] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jul 21 05:31:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:39.154939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe81d6-d81d-4e12-8ba1-b43d85a2496c None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:39.177075 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725294f9-a7a8-4241-9501-d056a44a2465 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:39.213319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e933c-e9e7-4ac8-9f44-ea87499ac947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:39.214161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e933c-e9e7-4ac8-9f44-ea87499ac947 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:39.214451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e933c-e9e7-4ac8-9f44-ea87499ac947 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:39.214789 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e933c-e9e7-4ac8-9f44-ea87499ac947 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:39.215515 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e933c-e9e7-4ac8-9f44-ea87499ac947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:39.217010 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e933c-e9e7-4ac8-9f44-ea87499ac947 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:39.223983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 451/895] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jul 21 05:31:39 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:31:39.269138 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe81d6-d81d-4e12-8ba1-b43d85a2496c None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:39.302870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:39.309816 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72d4ef-1530-469c-854e-f7ed78d4f6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:39.310893 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72d4ef-1530-469c-854e-f7ed78d4f6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:39.311410 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72d4ef-1530-469c-854e-f7ed78d4f6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:39.311892 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72d4ef-1530-469c-854e-f7ed78d4f6c5 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:39.312966 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72d4ef-1530-469c-854e-f7ed78d4f6c5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:39.313477 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72d4ef-1530-469c-854e-f7ed78d4f6c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:39.315267 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72d4ef-1530-469c-854e-f7ed78d4f6c5 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:39.335878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 445/896] 10.222.0.44 () {56 vars in 1149 bytes} [Thu Jul 21 05:31:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:39.342137 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e933c-e9e7-4ac8-9f44-ea87499ac947 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:39.380466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:39.450949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e933c-e9e7-4ac8-9f44-ea87499ac947 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:39.485556 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d58e8-3f85-49df-9a10-00bb61fe7c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:39.486596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d58e8-3f85-49df-9a10-00bb61fe7c31 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:39.487134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d58e8-3f85-49df-9a10-00bb61fe7c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:39.487620 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d58e8-3f85-49df-9a10-00bb61fe7c31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:39.560118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d58e8-3f85-49df-9a10-00bb61fe7c31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:39.561604 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d58e8-3f85-49df-9a10-00bb61fe7c31 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:39.597532 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:39.623466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996304be-d192-44f9-ac20-3ddb410b2691 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:39.623791 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996304be-d192-44f9-ac20-3ddb410b2691 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:39.624028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996304be-d192-44f9-ac20-3ddb410b2691 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:39.629368 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 446/897] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:39.631078 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cda02-4240-4322-a7f5-f3ccf3cd23fd None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:39.631378 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cda02-4240-4322-a7f5-f3ccf3cd23fd None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:39.631762 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cda02-4240-4322-a7f5-f3ccf3cd23fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:39.656955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-585cda02-4240-4322-a7f5-f3ccf3cd23fd None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:39.658439 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-585cda02-4240-4322-a7f5-f3ccf3cd23fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:39.684836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 452/898] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:39 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 05:31:39.690722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d58e8-3f85-49df-9a10-00bb61fe7c31 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:39.740715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 447/899] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:39.742026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf171b-f17a-47da-aeb9-a7fcb41d2f7e None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:39.742311 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf171b-f17a-47da-aeb9-a7fcb41d2f7e None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:39.742586 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf171b-f17a-47da-aeb9-a7fcb41d2f7e None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:39.744026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 448/900] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:39.749039 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc4153-240c-41af-9039-16725fb86f7c None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:39.749324 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc4153-240c-41af-9039-16725fb86f7c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:39.749599 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc4153-240c-41af-9039-16725fb86f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:39.765965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-77cc4153-240c-41af-9039-16725fb86f7c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:39.767221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-77cc4153-240c-41af-9039-16725fb86f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:39.800068 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d58e8-3f85-49df-9a10-00bb61fe7c31 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:39.842171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 449/901] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:39.843426 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc4153-240c-41af-9039-16725fb86f7c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:39.852751 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d154dc-69e0-463d-89bf-c3782bdac633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:39.853839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d154dc-69e0-463d-89bf-c3782bdac633 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:39.854325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d154dc-69e0-463d-89bf-c3782bdac633 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:39.854816 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d154dc-69e0-463d-89bf-c3782bdac633 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:39.858206 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:39.868860 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d154dc-69e0-463d-89bf-c3782bdac633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:39.870656 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d154dc-69e0-463d-89bf-c3782bdac633 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:39.888161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-17d154dc-69e0-463d-89bf-c3782bdac633 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:39.888161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:39.888161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:39.888161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:39.888161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:39.888161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:39.888161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:39.888161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:39.888161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:39.888161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:39.888161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:39.888161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:39.888161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:39.888161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:39.888161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:39.889115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:39.889115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:39.889115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:39.889115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:39.889115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:39.889115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:39.889115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:39.889115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:39.889115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:39.889115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:39.889115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:39.889115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:39.889115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:39.889115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:39.889115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:39.889115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:39.889115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:39.890034 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:39.890034 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:39.890034 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:39.890034 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:39.894036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 453/902] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:39.894875 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d154dc-69e0-463d-89bf-c3782bdac633 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:39.900295 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:39.954052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc4153-240c-41af-9039-16725fb86f7c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:39.992064 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b13e2-2a5d-4d88-a9b9-faedc438075a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:39.992808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b13e2-2a5d-4d88-a9b9-faedc438075a None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:39.993103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b13e2-2a5d-4d88-a9b9-faedc438075a None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:39.993375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b13e2-2a5d-4d88-a9b9-faedc438075a None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:39.994100 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b13e2-2a5d-4d88-a9b9-faedc438075a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:39.995515 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b13e2-2a5d-4d88-a9b9-faedc438075a None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:39.996663 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d154dc-69e0-463d-89bf-c3782bdac633 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:40.009645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 450/903] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jul 21 05:31:39 2022] POST /identity/v3/services => generated 241 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:31:40.010511 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b13e2-2a5d-4d88-a9b9-faedc438075a None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:40.046513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b91ad1-e5cc-45d6-83cc-b90ed6d58bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:40.047448 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b91ad1-e5cc-45d6-83cc-b90ed6d58bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:40.047736 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b91ad1-e5cc-45d6-83cc-b90ed6d58bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:40.048009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b91ad1-e5cc-45d6-83cc-b90ed6d58bc3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:40.048685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b91ad1-e5cc-45d6-83cc-b90ed6d58bc3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:40.050119 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b91ad1-e5cc-45d6-83cc-b90ed6d58bc3 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:40.063579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-19b91ad1-e5cc-45d6-83cc-b90ed6d58bc3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 05:31:40.063579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:40.063579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:40.063579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:40.063579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:40.063579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:40.063579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:40.063579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:40.063579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 05:31:40.063579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:40.063579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:40.063579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:40.063579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:40.063579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:40.063579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:40.063579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:40.064649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:40.064649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:40.064649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:40.064649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:40.064649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:40.064649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:40.064649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:40.064649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:40.064649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:40.064649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:40.064649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:40.064649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:40.064649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:40.064649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:40.064649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:40.064649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:40.065490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:40.065490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:40.065490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:40.065490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:40.065490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:40.065490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:40.065490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:40.065490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:40.065490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:40.065490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:40.065490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:40.065490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:40.065490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:40.065490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 05:31:40.065490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:40.070588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 454/904] 10.222.0.44 () {56 vars in 1132 bytes} [Thu Jul 21 05:31:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:40.071665 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b91ad1-e5cc-45d6-83cc-b90ed6d58bc3 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:40.136754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:40.150881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b13e2-2a5d-4d88-a9b9-faedc438075a None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:40.163616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:40.204417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0a3cd-f15c-4b94-a535-7bdd0c1a4047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:40.205090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0a3cd-f15c-4b94-a535-7bdd0c1a4047 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:40.205367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0a3cd-f15c-4b94-a535-7bdd0c1a4047 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:40.205634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0a3cd-f15c-4b94-a535-7bdd0c1a4047 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:40.206363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0a3cd-f15c-4b94-a535-7bdd0c1a4047 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:40.206661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0a3cd-f15c-4b94-a535-7bdd0c1a4047 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:40.208100 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0a3cd-f15c-4b94-a535-7bdd0c1a4047 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:40.209304 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b91ad1-e5cc-45d6-83cc-b90ed6d58bc3 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:40.221451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 451/905] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:40.227160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0a3cd-f15c-4b94-a535-7bdd0c1a4047 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:40.246286 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcaf17-6273-4ab1-8c0e-6090eba6e549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:40.246986 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcaf17-6273-4ab1-8c0e-6090eba6e549 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:40.247298 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcaf17-6273-4ab1-8c0e-6090eba6e549 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:40.247568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcaf17-6273-4ab1-8c0e-6090eba6e549 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:40.248268 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efcaf17-6273-4ab1-8c0e-6090eba6e549 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:40.248569 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efcaf17-6273-4ab1-8c0e-6090eba6e549 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:40.249997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efcaf17-6273-4ab1-8c0e-6090eba6e549 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:40.259053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 455/906] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jul 21 05:31:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:40.261169 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcaf17-6273-4ab1-8c0e-6090eba6e549 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:40.347438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0a3cd-f15c-4b94-a535-7bdd0c1a4047 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:40.374308 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcaf17-6273-4ab1-8c0e-6090eba6e549 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:40.391366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:40.394513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f64313-55ce-4565-abf5-a9fabbeaeec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:40.395756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f64313-55ce-4565-abf5-a9fabbeaeec0 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:40.396117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f64313-55ce-4565-abf5-a9fabbeaeec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:40.396490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f64313-55ce-4565-abf5-a9fabbeaeec0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:40.405130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f64313-55ce-4565-abf5-a9fabbeaeec0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:40.407289 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f64313-55ce-4565-abf5-a9fabbeaeec0 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:40.419303 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715bd3a-c125-4c5b-b578-25477511b568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:40.420316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715bd3a-c125-4c5b-b578-25477511b568 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:40.420771 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715bd3a-c125-4c5b-b578-25477511b568 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:40.421216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715bd3a-c125-4c5b-b578-25477511b568 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:40.422184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d715bd3a-c125-4c5b-b578-25477511b568 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:40.422670 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d715bd3a-c125-4c5b-b578-25477511b568 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:40.424374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-b5f64313-55ce-4565-abf5-a9fabbeaeec0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:40.424374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:40.424374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:40.424374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:40.424374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:40.424374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:40.424374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:40.424374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:40.424374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:40.424374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:40.424374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:40.424374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:40.424374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:40.424374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:40.424374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:40.425266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:40.425266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:40.425266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:40.425266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:40.425266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:40.425266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:40.425266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:40.425266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:40.425266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:40.425266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:40.425266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:40.425266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:40.425266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:40.425266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:40.425266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:40.425266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:40.425266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:40.426203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:40.426203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:40.426203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:40.426203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:40.426203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:40.426203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:40.426203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:40.426203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:40.427105 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d715bd3a-c125-4c5b-b578-25477511b568 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:40.430938 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 452/907] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:40.432665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f64313-55ce-4565-abf5-a9fabbeaeec0 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:40.436154 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 456/908] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jul 21 05:31:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:40.447078 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715bd3a-c125-4c5b-b578-25477511b568 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:40.474113 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:40.559952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715bd3a-c125-4c5b-b578-25477511b568 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:40.570519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f64313-55ce-4565-abf5-a9fabbeaeec0 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:40.603646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df3e4e-fad1-401c-b224-4fbad7c256a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:40.604322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df3e4e-fad1-401c-b224-4fbad7c256a9 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:40.604607 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df3e4e-fad1-401c-b224-4fbad7c256a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:40.604879 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df3e4e-fad1-401c-b224-4fbad7c256a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:40.605770 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df3e4e-fad1-401c-b224-4fbad7c256a9 None admin] RBAC: Adding query filter params (interface=public, service_id=07b0982a9ea849ba9575357ec8fcf1f7, region_id=RegionOne) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:40.606081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df3e4e-fad1-401c-b224-4fbad7c256a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:40.606521 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c69e21-b15d-4e5a-8ded-b87c57e137d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:40.607389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c69e21-b15d-4e5a-8ded-b87c57e137d8 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:40.607571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df3e4e-fad1-401c-b224-4fbad7c256a9 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:40.607977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c69e21-b15d-4e5a-8ded-b87c57e137d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:40.608418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c69e21-b15d-4e5a-8ded-b87c57e137d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:40.609350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c69e21-b15d-4e5a-8ded-b87c57e137d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:40.609813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c69e21-b15d-4e5a-8ded-b87c57e137d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:40.611505 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c69e21-b15d-4e5a-8ded-b87c57e137d8 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:40.615156 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 457/909] 10.222.0.44 () {56 vars in 1251 bytes} [Thu Jul 21 05:31:40 2022] GET /identity/v3/endpoints?service_id=07b0982a9ea849ba9575357ec8fcf1f7&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:40.619421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df3e4e-fad1-401c-b224-4fbad7c256a9 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:40.631058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 453/910] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:40.638421 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c69e21-b15d-4e5a-8ded-b87c57e137d8 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:40.678595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:40.734053 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:40.737936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df3e4e-fad1-401c-b224-4fbad7c256a9 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:40.748751 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c69e21-b15d-4e5a-8ded-b87c57e137d8 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:40.772865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f3989-4c9c-4464-9130-7dd169f85c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:40.773698 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f3989-4c9c-4464-9130-7dd169f85c4f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:40.774087 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f3989-4c9c-4464-9130-7dd169f85c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:40.774488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f3989-4c9c-4464-9130-7dd169f85c4f None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:40.775272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f3989-4c9c-4464-9130-7dd169f85c4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:40.776757 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f3989-4c9c-4464-9130-7dd169f85c4f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:40.788194 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 458/911] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jul 21 05:31:40 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:31:40.793875 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e0a0f-d350-4df9-aecb-95e6facc8c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:40.794573 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e0a0f-d350-4df9-aecb-95e6facc8c33 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:40.794859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e0a0f-d350-4df9-aecb-95e6facc8c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:40.795171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e0a0f-d350-4df9-aecb-95e6facc8c33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:40.796042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e0a0f-d350-4df9-aecb-95e6facc8c33 None admin] RBAC: Adding query filter params (role.id=545b77cdab164cd29c9b639b341c4018, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=4d04998b636649d98103b1cd5f3a0f7c) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:40.796351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e0a0f-d350-4df9-aecb-95e6facc8c33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:40.797843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e0a0f-d350-4df9-aecb-95e6facc8c33 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:40.811298 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 454/912] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:40 2022] GET /identity/v3/role_assignments?user.id=4d04998b636649d98103b1cd5f3a0f7c&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=545b77cdab164cd29c9b639b341c4018 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:41.368444 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093b607-0b17-4c91-9007-76da94c82256 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:41.369205 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093b607-0b17-4c91-9007-76da94c82256 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:41.369205 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093b607-0b17-4c91-9007-76da94c82256 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:41.370520 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 459/913] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:41.375929 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a80683-056b-4b8f-8a4c-1320f3c6fded None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:41.376733 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a80683-056b-4b8f-8a4c-1320f3c6fded None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:41.377228 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a80683-056b-4b8f-8a4c-1320f3c6fded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:41.394536 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-73a80683-056b-4b8f-8a4c-1320f3c6fded None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:41.396042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-73a80683-056b-4b8f-8a4c-1320f3c6fded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:41.479095 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 455/914] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:41.501619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093b607-0b17-4c91-9007-76da94c82256 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:41.604873 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093b607-0b17-4c91-9007-76da94c82256 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:41.607248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:41.639440 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc26f8-8949-4753-8342-5f093fd65b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:41.640175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc26f8-8949-4753-8342-5f093fd65b11 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:41.640460 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc26f8-8949-4753-8342-5f093fd65b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:41.640734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc26f8-8949-4753-8342-5f093fd65b11 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:41.641387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc26f8-8949-4753-8342-5f093fd65b11 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:41.642771 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc26f8-8949-4753-8342-5f093fd65b11 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:41.660364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-fefc26f8-8949-4753-8342-5f093fd65b11 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 05:31:41.660364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:41.660364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:41.660364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:41.660364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:41.660364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:41.660364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:41.660364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:41.660364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 05:31:41.660364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:41.660364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:41.660364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:41.660364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:41.660364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:41.660364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:41.660364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:41.660364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:41.661372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:41.661372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:41.661372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:41.661372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:41.661372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:41.661372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:41.661372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:41.661372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:41.661372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:41.661372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:41.661372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:41.661372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:41.661372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:41.661372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:41.661372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:41.661372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:41.662209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:41.662209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:41.662209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:41.662209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:41.662209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:41.662209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:41.662209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:41.662209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:41.662209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:41.662209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:41.662209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:41.662209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:41.662209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 21 05:31:41.662209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:41.662209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 460/915] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:31:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:41.670687 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a80683-056b-4b8f-8a4c-1320f3c6fded None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:41.752390 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:41.771878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a80683-056b-4b8f-8a4c-1320f3c6fded None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:41.814131 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb66e2-5730-420c-831e-d54c078f3662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:41.814812 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb66e2-5730-420c-831e-d54c078f3662 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:41.815121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb66e2-5730-420c-831e-d54c078f3662 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:41.815398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb66e2-5730-420c-831e-d54c078f3662 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:41.816114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb66e2-5730-420c-831e-d54c078f3662 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:41.816420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb66e2-5730-420c-831e-d54c078f3662 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:41.817804 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb66e2-5730-420c-831e-d54c078f3662 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:41.823146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 456/916] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:31:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:41.828806 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc26f8-8949-4753-8342-5f093fd65b11 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:41.936690 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc26f8-8949-4753-8342-5f093fd65b11 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:41.973025 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466c4fd-a2cb-4801-8f17-aefc73ba949e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:41.973712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466c4fd-a2cb-4801-8f17-aefc73ba949e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:41.973995 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466c4fd-a2cb-4801-8f17-aefc73ba949e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:41.974274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466c4fd-a2cb-4801-8f17-aefc73ba949e None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:41.974997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466c4fd-a2cb-4801-8f17-aefc73ba949e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:41.975410 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466c4fd-a2cb-4801-8f17-aefc73ba949e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:41.976839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466c4fd-a2cb-4801-8f17-aefc73ba949e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:41.983218 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 461/917] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jul 21 05:31:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:41.989198 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb66e2-5730-420c-831e-d54c078f3662 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:42.091340 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb66e2-5730-420c-831e-d54c078f3662 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:42.113378 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af32e56-60e5-4858-b89e-87c459d1e103 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:42.113708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af32e56-60e5-4858-b89e-87c459d1e103 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:42.113936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af32e56-60e5-4858-b89e-87c459d1e103 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:42.115640 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 462/918] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:31:42.120488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91585498-7fb9-4a76-8504-506af9126302 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:42.120772 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91585498-7fb9-4a76-8504-506af9126302 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:42.121067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91585498-7fb9-4a76-8504-506af9126302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:42.133357 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ab9ac-5108-4551-b3b6-aaecc7675f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:42.134021 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ab9ac-5108-4551-b3b6-aaecc7675f83 None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:42.134301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ab9ac-5108-4551-b3b6-aaecc7675f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:42.134570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ab9ac-5108-4551-b3b6-aaecc7675f83 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:42.135352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ab9ac-5108-4551-b3b6-aaecc7675f83 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:42.136772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ab9ac-5108-4551-b3b6-aaecc7675f83 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:42.143151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-91585498-7fb9-4a76-8504-506af9126302 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:42.145352 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-91585498-7fb9-4a76-8504-506af9126302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:42.160402 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 457/919] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jul 21 05:31:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:31:42.186455 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:42.232116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 463/920] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:42.255754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ab9ac-5108-4551-b3b6-aaecc7675f83 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:42.258820 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:42.364821 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ab9ac-5108-4551-b3b6-aaecc7675f83 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:42.399730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd66675-727a-4136-adee-0eacec438f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:42.400546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd66675-727a-4136-adee-0eacec438f9c None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:42.400824 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd66675-727a-4136-adee-0eacec438f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:42.401094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd66675-727a-4136-adee-0eacec438f9c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:42.401753 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd66675-727a-4136-adee-0eacec438f9c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:42.403157 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd66675-727a-4136-adee-0eacec438f9c None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:42.417017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-6fd66675-727a-4136-adee-0eacec438f9c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 05:31:42.417017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:42.417017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:42.417017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:42.417017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:42.417017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:42.417017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:42.417017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:42.417017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 05:31:42.417017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:42.417017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:42.417017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:42.417017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:42.417017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:42.417017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:42.417017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:42.417017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:42.418074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:42.418074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:42.418074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:42.418074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:42.418074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:42.418074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:42.418074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:42.418074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:42.418074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:42.418074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:42.418074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:42.418074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:42.418074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:42.418074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:42.418074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:42.418074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:42.418922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:42.418922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:42.418922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:42.418922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:42.418922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:42.418922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:42.418922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:42.418922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:42.418922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:42.418922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:42.418922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:42.418922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:42.418922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 05:31:42.418922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:42.418922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 458/921] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jul 21 05:31:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:42.424759 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91585498-7fb9-4a76-8504-506af9126302 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:42.524322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91585498-7fb9-4a76-8504-506af9126302 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:42.558672 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44eafa8-a6de-4686-aa6b-202222457e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:42.559433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eafa8-a6de-4686-aa6b-202222457e33 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:42.559711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eafa8-a6de-4686-aa6b-202222457e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:42.560026 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44eafa8-a6de-4686-aa6b-202222457e33 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:42.560735 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44eafa8-a6de-4686-aa6b-202222457e33 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:42.561039 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44eafa8-a6de-4686-aa6b-202222457e33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:42.562412 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44eafa8-a6de-4686-aa6b-202222457e33 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:42.568064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 464/922] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jul 21 05:31:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:42.573825 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd66675-727a-4136-adee-0eacec438f9c None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:42.642149 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:42.679467 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd66675-727a-4136-adee-0eacec438f9c None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:42.716689 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bbcf2-5b7a-49be-b61a-248c2894347f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:42.717771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bbcf2-5b7a-49be-b61a-248c2894347f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:42.718261 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bbcf2-5b7a-49be-b61a-248c2894347f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:42.718728 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bbcf2-5b7a-49be-b61a-248c2894347f None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:42.719681 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bbcf2-5b7a-49be-b61a-248c2894347f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:42.720274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bbcf2-5b7a-49be-b61a-248c2894347f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:42.721907 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bbcf2-5b7a-49be-b61a-248c2894347f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:42.728012 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 459/923] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jul 21 05:31:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:42.737232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44eafa8-a6de-4686-aa6b-202222457e33 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:42.763007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:42.839827 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44eafa8-a6de-4686-aa6b-202222457e33 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:42.874974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b03d32-abe0-4fd8-b157-ed90d5722d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:42.875722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b03d32-abe0-4fd8-b157-ed90d5722d7c None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:42.876001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b03d32-abe0-4fd8-b157-ed90d5722d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:42.876273 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b03d32-abe0-4fd8-b157-ed90d5722d7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:42.877115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b03d32-abe0-4fd8-b157-ed90d5722d7c None admin] RBAC: Adding query filter params (interface=public, service_id=3f47157483f44de98b041a1f4e12f212, region_id=RegionOne) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:42.877428 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b03d32-abe0-4fd8-b157-ed90d5722d7c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:42.878800 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b03d32-abe0-4fd8-b157-ed90d5722d7c None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:42.884421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 465/924] 10.222.0.44 () {56 vars in 1251 bytes} [Thu Jul 21 05:31:42 2022] GET /identity/v3/endpoints?service_id=3f47157483f44de98b041a1f4e12f212&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:42.891161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bbcf2-5b7a-49be-b61a-248c2894347f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:42.906690 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c6150-e60f-4fe2-a23a-40d69e4f7993 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:42.907054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c6150-e60f-4fe2-a23a-40d69e4f7993 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:42.907271 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c6150-e60f-4fe2-a23a-40d69e4f7993 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:42.908938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 466/925] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:31:42.915554 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6daca1-ecf6-49c5-8571-56809417b23b None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:42.916022 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6daca1-ecf6-49c5-8571-56809417b23b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:42.916507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6daca1-ecf6-49c5-8571-56809417b23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:42.934037 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-bd6daca1-ecf6-49c5-8571-56809417b23b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:42.935509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-bd6daca1-ecf6-49c5-8571-56809417b23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:43.000548 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bbcf2-5b7a-49be-b61a-248c2894347f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:43.005697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 467/926] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:43.007402 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657de084-652c-47d3-a25a-d9b0821914b8 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:43.008247 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657de084-652c-47d3-a25a-d9b0821914b8 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:43.008685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657de084-652c-47d3-a25a-d9b0821914b8 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:43.010749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 468/927] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:43.015793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a841dde-8a34-4254-8c07-d0271bb96209 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:43.016081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a841dde-8a34-4254-8c07-d0271bb96209 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:43.016360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a841dde-8a34-4254-8c07-d0271bb96209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:43.030813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:43.034919 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-5a841dde-8a34-4254-8c07-d0271bb96209 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:43.036177 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5a841dde-8a34-4254-8c07-d0271bb96209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:43.038175 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176dfc75-ef03-4c29-8143-963dc2f0e306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:43.040127 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dfc75-ef03-4c29-8143-963dc2f0e306 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:43.040419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dfc75-ef03-4c29-8143-963dc2f0e306 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:43.040694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176dfc75-ef03-4c29-8143-963dc2f0e306 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:43.041388 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176dfc75-ef03-4c29-8143-963dc2f0e306 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:43.042755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176dfc75-ef03-4c29-8143-963dc2f0e306 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:43.048256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:43.059061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 460/928] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jul 21 05:31:42 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:31:43.059061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176dfc75-ef03-4c29-8143-963dc2f0e306 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:43.143289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 469/929] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:43.170392 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a841dde-8a34-4254-8c07-d0271bb96209 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:43.194340 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176dfc75-ef03-4c29-8143-963dc2f0e306 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:43.242171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9fbec6-fb2d-415e-b67f-608df048504e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:43.242998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fbec6-fb2d-415e-b67f-608df048504e None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:43.243308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fbec6-fb2d-415e-b67f-608df048504e None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:43.243584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fbec6-fb2d-415e-b67f-608df048504e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:43.258112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9fbec6-fb2d-415e-b67f-608df048504e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:43.266403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-ed9fbec6-fb2d-415e-b67f-608df048504e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 05:31:43.266403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:43.266403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:43.266403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:43.266403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:43.266403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:43.266403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:43.266403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:43.266403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:43.266403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:43.266403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:43.266403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:43.266403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:43.266403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:43.266403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:43.266403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:43.267444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:43.267444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:43.267444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:43.267444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:43.267444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:43.267444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:43.267444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:43.267444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:43.267444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:43.267444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:43.267444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:43.267444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:43.267444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:43.267444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:43.267444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:43.267444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:43.268323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:43.268323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:43.268323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:43.268323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:43.268323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:43.268323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:43.268323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:43.268323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:43.268323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:43.268323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:43.268323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 05:31:43.268323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:43.268323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 461/930] 10.222.0.44 () {56 vars in 1120 bytes} [Thu Jul 21 05:31:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:43.273802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9fbec6-fb2d-415e-b67f-608df048504e None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:43.290841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a841dde-8a34-4254-8c07-d0271bb96209 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:43.325789 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261eb234-ae2c-44ef-ac4f-bd55c87926c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:43.326628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261eb234-ae2c-44ef-ac4f-bd55c87926c3 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:43.326913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261eb234-ae2c-44ef-ac4f-bd55c87926c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:43.327234 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261eb234-ae2c-44ef-ac4f-bd55c87926c3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:43.327902 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261eb234-ae2c-44ef-ac4f-bd55c87926c3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:43.329397 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261eb234-ae2c-44ef-ac4f-bd55c87926c3 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:43.340085 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:43.342644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-261eb234-ae2c-44ef-ac4f-bd55c87926c3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 05:31:43.342644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:43.342644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:43.342644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:43.342644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:43.342644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:43.342644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:43.342644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:43.342644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 05:31:43.342644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:43.342644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:43.342644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:43.342644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:43.342644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:43.342644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:43.342644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:43.343661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:43.343661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:43.343661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:43.343661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:43.343661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:43.343661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:43.343661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:43.343661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:43.343661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:43.343661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:43.343661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:43.343661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:43.343661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:43.343661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:43.343661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:43.343661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:43.344525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:43.344525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:43.344525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:43.344525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:43.344525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:43.344525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:43.344525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:43.344525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:43.344525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:43.344525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:43.344525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:43.344525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:43.344525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:43.344525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 21 05:31:43.344525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:43.344525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 470/931] 10.222.0.44 () {56 vars in 1132 bytes} [Thu Jul 21 05:31:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:43.351486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261eb234-ae2c-44ef-ac4f-bd55c87926c3 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:43.368854 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:43.405552 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9fbec6-fb2d-415e-b67f-608df048504e None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:43.454819 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e9e80-5132-4c5c-a4cb-7099257423f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:43.456054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e9e80-5132-4c5c-a4cb-7099257423f8 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:43.456539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e9e80-5132-4c5c-a4cb-7099257423f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:43.457087 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e9e80-5132-4c5c-a4cb-7099257423f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:43.458106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e9e80-5132-4c5c-a4cb-7099257423f8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:43.458609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e9e80-5132-4c5c-a4cb-7099257423f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:43.460295 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e9e80-5132-4c5c-a4cb-7099257423f8 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:43.469701 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 462/932] 10.222.0.44 () {56 vars in 1115 bytes} [Thu Jul 21 05:31:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:43.474878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e9e80-5132-4c5c-a4cb-7099257423f8 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:43.483821 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261eb234-ae2c-44ef-ac4f-bd55c87926c3 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:43.518391 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fa43c-ac2c-46e3-adb0-3eca4e5bfa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:43.519098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fa43c-ac2c-46e3-adb0-3eca4e5bfa74 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:43.519381 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fa43c-ac2c-46e3-adb0-3eca4e5bfa74 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:43.519653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fa43c-ac2c-46e3-adb0-3eca4e5bfa74 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:43.520346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6fa43c-ac2c-46e3-adb0-3eca4e5bfa74 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:43.520649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6fa43c-ac2c-46e3-adb0-3eca4e5bfa74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:43.522063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6fa43c-ac2c-46e3-adb0-3eca4e5bfa74 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:43.536081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 471/933] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jul 21 05:31:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:43.536850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fa43c-ac2c-46e3-adb0-3eca4e5bfa74 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:43.611174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:43.611510 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e9e80-5132-4c5c-a4cb-7099257423f8 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:43.633330 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:43.638070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fa43c-ac2c-46e3-adb0-3eca4e5bfa74 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:43.659294 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67051f6-d049-4f47-91c4-62b3c5e3fdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:43.660891 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67051f6-d049-4f47-91c4-62b3c5e3fdb1 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:43.661382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67051f6-d049-4f47-91c4-62b3c5e3fdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:43.662861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67051f6-d049-4f47-91c4-62b3c5e3fdb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:43.674519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67051f6-d049-4f47-91c4-62b3c5e3fdb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:43.676567 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67051f6-d049-4f47-91c4-62b3c5e3fdb1 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:43.676852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427d328-800c-4a4d-a802-fe805be5443f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:43.677588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427d328-800c-4a4d-a802-fe805be5443f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:43.677939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427d328-800c-4a4d-a802-fe805be5443f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:43.678364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427d328-800c-4a4d-a802-fe805be5443f None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:43.679253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427d328-800c-4a4d-a802-fe805be5443f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:43.679625 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427d328-800c-4a4d-a802-fe805be5443f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:43.681189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427d328-800c-4a4d-a802-fe805be5443f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:43.687056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 472/934] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jul 21 05:31:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:43.692012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427d328-800c-4a4d-a802-fe805be5443f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:43.699977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-a67051f6-d049-4f47-91c4-62b3c5e3fdb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:43.699977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:43.699977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:43.699977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:43.699977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:43.699977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:43.699977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:43.699977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:43.699977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:43.699977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:43.699977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:43.699977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:43.699977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:43.699977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:43.699977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:43.700864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:43.700864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:43.700864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:43.700864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:43.700864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:43.700864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:43.700864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:43.700864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:43.700864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:43.700864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:43.700864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:43.700864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:43.700864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:43.700864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:43.700864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:43.700864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:43.700864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:43.701860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:43.701860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:43.701860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:43.701860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:43.706443 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 463/935] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:43.707086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67051f6-d049-4f47-91c4-62b3c5e3fdb1 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:43.803416 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427d328-800c-4a4d-a802-fe805be5443f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:43.808576 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67051f6-d049-4f47-91c4-62b3c5e3fdb1 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:43.843769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f116ef7-4732-41e1-bd50-66874e8b08a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:43.844811 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f116ef7-4732-41e1-bd50-66874e8b08a1 None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:43.845575 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f116ef7-4732-41e1-bd50-66874e8b08a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:43.846066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f116ef7-4732-41e1-bd50-66874e8b08a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:43.847687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f116ef7-4732-41e1-bd50-66874e8b08a1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:43.849662 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f116ef7-4732-41e1-bd50-66874e8b08a1 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:43.852342 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a16fa-1082-4120-84dd-79b877990f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:43.853027 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a16fa-1082-4120-84dd-79b877990f76 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:43.853310 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a16fa-1082-4120-84dd-79b877990f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:43.853587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a16fa-1082-4120-84dd-79b877990f76 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:43.854561 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a16fa-1082-4120-84dd-79b877990f76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:43.854857 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a16fa-1082-4120-84dd-79b877990f76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:43.856282 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a16fa-1082-4120-84dd-79b877990f76 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:43.860101 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:43.876424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 464/936] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:43.878659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 473/937] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jul 21 05:31:43 2022] POST /identity/v3/endpoints => generated 347 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:31:43.878789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a16fa-1082-4120-84dd-79b877990f76 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:43.932031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:44.003455 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a16fa-1082-4120-84dd-79b877990f76 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:44.049760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24407618-e364-46ab-897b-e837048c799c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:44.050591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407618-e364-46ab-897b-e837048c799c None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:44.050871 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407618-e364-46ab-897b-e837048c799c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:44.051246 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24407618-e364-46ab-897b-e837048c799c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:44.058789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24407618-e364-46ab-897b-e837048c799c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:44.060284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24407618-e364-46ab-897b-e837048c799c None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:44.075887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-24407618-e364-46ab-897b-e837048c799c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 05:31:44.075887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:44.075887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:44.075887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:44.075887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:44.075887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:44.075887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:44.075887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:44.075887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:44.075887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:44.075887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:44.075887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:44.075887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:44.075887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:44.075887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:44.075887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:44.076909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:44.077863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:44.077863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:44.077863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:44.077863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:44.077863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:44.077863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:44.077863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:44.077863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:44.077863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 05:31:44.077863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:44.077863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 465/938] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jul 21 05:31:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:44.085199 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f116ef7-4732-41e1-bd50-66874e8b08a1 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:44.188883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f116ef7-4732-41e1-bd50-66874e8b08a1 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:44.226911 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9188dc-67c0-41e2-8757-2192dfde4d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:44.227668 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9188dc-67c0-41e2-8757-2192dfde4d36 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:44.227948 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9188dc-67c0-41e2-8757-2192dfde4d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:44.228218 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9188dc-67c0-41e2-8757-2192dfde4d36 None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:44.229056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9188dc-67c0-41e2-8757-2192dfde4d36 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:44.229360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9188dc-67c0-41e2-8757-2192dfde4d36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:44.230806 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9188dc-67c0-41e2-8757-2192dfde4d36 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:44.249479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 474/939] 10.222.0.44 () {56 vars in 1149 bytes} [Thu Jul 21 05:31:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:44.255612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24407618-e364-46ab-897b-e837048c799c None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:44.306714 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0ba99-8fa9-4eda-9fb1-d2b552db898a None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:44.307087 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0ba99-8fa9-4eda-9fb1-d2b552db898a None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:44.307317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0ba99-8fa9-4eda-9fb1-d2b552db898a None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:44.309068 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 475/940] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:44.316462 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae08a2-5b6f-4527-abdb-f77229850106 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:44.316870 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae08a2-5b6f-4527-abdb-f77229850106 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:44.317414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae08a2-5b6f-4527-abdb-f77229850106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:44.334988 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-b2ae08a2-5b6f-4527-abdb-f77229850106 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:44.336344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b2ae08a2-5b6f-4527-abdb-f77229850106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:44.346312 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:44.361913 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24407618-e364-46ab-897b-e837048c799c None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:44.393548 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:44.409086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c98c22-3169-44c1-9ea7-d18b98f419d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:44.409904 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c98c22-3169-44c1-9ea7-d18b98f419d2 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:44.410185 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c98c22-3169-44c1-9ea7-d18b98f419d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:44.410546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c98c22-3169-44c1-9ea7-d18b98f419d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:44.423150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 476/941] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:44.431248 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c98c22-3169-44c1-9ea7-d18b98f419d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:44.432977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c98c22-3169-44c1-9ea7-d18b98f419d2 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:44.447455 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae08a2-5b6f-4527-abdb-f77229850106 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:44.456806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-57c98c22-3169-44c1-9ea7-d18b98f419d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:44.456806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:44.456806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:44.456806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:44.456806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:44.456806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:44.456806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:44.456806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:44.456806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:44.456806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:44.456806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:44.456806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:44.456806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:44.456806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:44.456806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:44.457675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:44.457675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:44.457675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:44.457675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:44.457675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:44.457675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:44.457675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:44.457675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:44.457675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:44.457675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:44.457675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:44.457675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:44.457675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:44.457675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:44.457675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:44.457675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:44.457675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:44.458566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:44.458566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:44.458566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:44.458566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:44.458566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 466/942] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:44.464115 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c98c22-3169-44c1-9ea7-d18b98f419d2 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:44.550430 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae08a2-5b6f-4527-abdb-f77229850106 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:44.567650 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c98c22-3169-44c1-9ea7-d18b98f419d2 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:44.586736 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac2e91-1b08-46ab-bc7f-abbe976a01ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:44.587799 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac2e91-1b08-46ab-bc7f-abbe976a01ee None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:44.588278 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac2e91-1b08-46ab-bc7f-abbe976a01ee None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:44.588767 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac2e91-1b08-46ab-bc7f-abbe976a01ee None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:44.589679 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac2e91-1b08-46ab-bc7f-abbe976a01ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:44.591361 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac2e91-1b08-46ab-bc7f-abbe976a01ee None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:44.592958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: WARNING keystone.server.flask.common [None req-e9ac2e91-1b08-46ab-bc7f-abbe976a01ee None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 21 05:31:44.603911 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb064b63-6aba-4b87-9272-fccf08741601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:44.604333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb064b63-6aba-4b87-9272-fccf08741601 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:44.604618 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb064b63-6aba-4b87-9272-fccf08741601 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:44.604895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb064b63-6aba-4b87-9272-fccf08741601 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:44.605769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb064b63-6aba-4b87-9272-fccf08741601 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:44.605995 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb064b63-6aba-4b87-9272-fccf08741601 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:44.607453 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb064b63-6aba-4b87-9272-fccf08741601 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:44.609992 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:44.616590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.sql.core [None req-e9ac2e91-1b08-46ab-bc7f-abbe976a01ee None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 05:31:44.616590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 05:31:44.616590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [parameters: {'id': 'bbb11bab98a8400e8d8aecb5bd2441de', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 05:31:44.616590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80391) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 05:31:44.627438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 467/943] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:44.628543 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb064b63-6aba-4b87-9272-fccf08741601 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:44.637375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-e9ac2e91-1b08-46ab-bc7f-abbe976a01ee None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bbb11bab98a8400e8d8aecb5bd2441de.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bbb11bab98a8400e8d8aecb5bd2441de. Jul 21 05:31:44.637375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:44.637375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 05:31:44.637375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 05:31:44.637375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 05:31:44.637375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 05:31:44.637375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 05:31:44.637375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 05:31:44.637375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 05:31:44.637375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application conn.query(q) Jul 21 05:31:44.637375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 05:31:44.637375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 05:31:44.637375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application result.read() Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 21 05:31:44.638342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 21 05:31:44.639484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 05:31:44.639484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application next(self.gen) Jul 21 05:31:44.639484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 05:31:44.639484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application yield resource Jul 21 05:31:44.639484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 05:31:44.639484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application next(self.gen) Jul 21 05:31:44.639484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 05:31:44.639484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 05:31:44.639484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 05:31:44.639484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 05:31:44.639484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 05:31:44.639484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise self.value Jul 21 05:31:44.639484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 05:31:44.639484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 05:31:44.639484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 05:31:44.639484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application session.commit() Jul 21 05:31:44.639484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 05:31:44.640413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 05:31:44.640413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 05:31:44.640413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 05:31:44.640413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 05:31:44.640413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self.session.flush() Jul 21 05:31:44.640413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 05:31:44.640413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 05:31:44.640413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 05:31:44.640413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 05:31:44.640413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 05:31:44.640413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application compat.raise_( Jul 21 05:31:44.640413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 05:31:44.640413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception Jul 21 05:31:44.640413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 05:31:44.640413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 05:31:44.640413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 05:31:44.640413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 05:31:44.641869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 05:31:44.641869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 05:31:44.641869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 05:31:44.641869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 05:31:44.641869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 21 05:31:44.641869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 21 05:31:44.641869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 05:31:44.641869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 05:31:44.641869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 05:31:44.641869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 05:31:44.641869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 05:31:44.641869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 05:31:44.641869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 05:31:44.641869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 05:31:44.641869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 05:31:44.642687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 05:31:44.642687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 05:31:44.642687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception Jul 21 05:31:44.642687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 05:31:44.642687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 05:31:44.642687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 05:31:44.642687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 05:31:44.642687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 05:31:44.642687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 05:31:44.642687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 05:31:44.642687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application conn.query(q) Jul 21 05:31:44.642687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 05:31:44.642687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 05:31:44.642687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 05:31:44.642687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application result.read() Jul 21 05:31:44.642687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 05:31:44.642687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 05:31:44.645021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 05:31:44.645021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 05:31:44.645021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 05:31:44.645021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 05:31:44.645021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 05:31:44.645021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 05:31:44.645021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 21 05:31:44.645021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 21 05:31:44.645021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [parameters: {'id': 'bbb11bab98a8400e8d8aecb5bd2441de', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 21 05:31:44.645021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 05:31:44.645021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:44.645021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:44.645021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:44.645021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:44.645021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 21 05:31:44.647048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 21 05:31:44.647048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 05:31:44.647048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 05:31:44.647048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 21 05:31:44.647048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:44.647048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:44.647048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:44.647048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:44.647048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:44.647048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:44.647048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:44.647048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:44.647048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:44.647048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:44.647048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:44.647048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:44.647048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:44.649316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:44.649316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 21 05:31:44.649316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 21 05:31:44.649316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:44.649316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:44.649316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 21 05:31:44.649316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 21 05:31:44.649316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bbb11bab98a8400e8d8aecb5bd2441de. Jul 21 05:31:44.649316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:44.649316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 477/944] 10.222.0.44 () {60 vars in 1140 bytes} [Thu Jul 21 05:31:44 2022] POST /identity/v3/projects => generated 262 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:44.649316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac2e91-1b08-46ab-bc7f-abbe976a01ee None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:44.700779 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:44.758212 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb064b63-6aba-4b87-9272-fccf08741601 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:44.775489 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac2e91-1b08-46ab-bc7f-abbe976a01ee None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:44.795903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854552e-6065-4359-973f-971f61ea68b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:44.797208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854552e-6065-4359-973f-971f61ea68b8 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:44.797910 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854552e-6065-4359-973f-971f61ea68b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:44.798428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854552e-6065-4359-973f-971f61ea68b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:44.806431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854552e-6065-4359-973f-971f61ea68b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:44.808308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854552e-6065-4359-973f-971f61ea68b8 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:44.814381 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77430a02-5c3a-41ca-aa38-6761e79aea41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:44.815216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77430a02-5c3a-41ca-aa38-6761e79aea41 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:44.815496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77430a02-5c3a-41ca-aa38-6761e79aea41 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:44.815786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77430a02-5c3a-41ca-aa38-6761e79aea41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:44.823223 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77430a02-5c3a-41ca-aa38-6761e79aea41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:44.824900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77430a02-5c3a-41ca-aa38-6761e79aea41 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:44.825236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-c854552e-6065-4359-973f-971f61ea68b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:44.825236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:44.825236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:44.825236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:44.825236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:44.825236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:44.825236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:44.825236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:44.825236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:44.825236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:44.825236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:44.825236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:44.825236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:44.825236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:44.825236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:44.826120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:44.826120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:44.826120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:44.826120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:44.826120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:44.826120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:44.826120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:44.826120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:44.826120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:44.826120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:44.826120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:44.826120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:44.826120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:44.826120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:44.826120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:44.826120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:44.826120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:44.827045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:44.827045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:44.827045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:44.827045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:44.827045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:44.827045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:44.827045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:44.827045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:44.834109 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 468/945] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:44.835280 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854552e-6065-4359-973f-971f61ea68b8 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:44.840457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-77430a02-5c3a-41ca-aa38-6761e79aea41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:44.840457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:44.840457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:44.840457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:44.840457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:44.840457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:44.840457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:44.840457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:44.840457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:44.840457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:44.840457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:44.840457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:44.840457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:44.840457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:44.840457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:44.841362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:44.841362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:44.841362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:44.841362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:44.841362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:44.841362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:44.841362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:44.841362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:44.841362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:44.841362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:44.841362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:44.841362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:44.841362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:44.841362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:44.841362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:44.841362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:44.841362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:44.842266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:44.842266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:44.842266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:44.842266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:44.842266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:44.842266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:44.842266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:44.842266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:44.842266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 478/946] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:44.847458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77430a02-5c3a-41ca-aa38-6761e79aea41 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:44.898748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:44.949221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854552e-6065-4359-973f-971f61ea68b8 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:44.978671 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:44.984771 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77430a02-5c3a-41ca-aa38-6761e79aea41 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:44.986193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e160d-4238-42c6-a9db-b71126410638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:44.987150 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e160d-4238-42c6-a9db-b71126410638 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:44.987639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e160d-4238-42c6-a9db-b71126410638 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:44.988143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e160d-4238-42c6-a9db-b71126410638 None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:44.989159 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e160d-4238-42c6-a9db-b71126410638 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:44.989674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e160d-4238-42c6-a9db-b71126410638 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:44.991392 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e160d-4238-42c6-a9db-b71126410638 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:45.010568 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 469/947] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:45.011805 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e160d-4238-42c6-a9db-b71126410638 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:45.027694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19035857-34d3-4373-bd54-4ec364af3018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:45.028420 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19035857-34d3-4373-bd54-4ec364af3018 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:45.028725 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19035857-34d3-4373-bd54-4ec364af3018 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:45.029431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19035857-34d3-4373-bd54-4ec364af3018 None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:45.030255 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19035857-34d3-4373-bd54-4ec364af3018 None admin] RBAC: Adding query filter params (name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:45.030569 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19035857-34d3-4373-bd54-4ec364af3018 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:45.032601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19035857-34d3-4373-bd54-4ec364af3018 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:45.067225 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 479/948] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jul 21 05:31:44 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:45.135376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e160d-4238-42c6-a9db-b71126410638 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:45.143158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b417b2b-22a1-43e9-98ae-0e4944294cd7 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:45.143503 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b417b2b-22a1-43e9-98ae-0e4944294cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:45.143760 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b417b2b-22a1-43e9-98ae-0e4944294cd7 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:45.145339 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 480/949] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:31:45.151298 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14485d4-66dc-4f28-8dea-b6dfa7c2989f None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:45.151610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14485d4-66dc-4f28-8dea-b6dfa7c2989f None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:45.151857 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14485d4-66dc-4f28-8dea-b6dfa7c2989f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:45.157270 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:45.172773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-b14485d4-66dc-4f28-8dea-b6dfa7c2989f None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:45.175173 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b14485d4-66dc-4f28-8dea-b6dfa7c2989f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:45.182333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4988432-3fb8-4ae7-be7f-56c1f5225c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:45.183734 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4988432-3fb8-4ae7-be7f-56c1f5225c42 None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:45.184226 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4988432-3fb8-4ae7-be7f-56c1f5225c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:45.184735 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4988432-3fb8-4ae7-be7f-56c1f5225c42 None admin] PATH_INFO: `/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/4d04998b636649d98103b1cd5f3a0f7c/roles/545b77cdab164cd29c9b639b341c4018` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:45.187336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4988432-3fb8-4ae7-be7f-56c1f5225c42 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b6f27b8d9b44d191210a24a9e04041, user_id=4d04998b636649d98103b1cd5f3a0f7c, role_id=545b77cdab164cd29c9b639b341c4018)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:45.189187 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4988432-3fb8-4ae7-be7f-56c1f5225c42 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:45.208735 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 470/950] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:31:45 2022] PUT /identity/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/4d04998b636649d98103b1cd5f3a0f7c/roles/545b77cdab164cd29c9b639b341c4018 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:31:45.262136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 481/951] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:45.284205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4988432-3fb8-4ae7-be7f-56c1f5225c42 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:45.396661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4988432-3fb8-4ae7-be7f-56c1f5225c42 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:45.433323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0878345-5fef-4ab2-b8ea-7b68ae085284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:45.434142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0878345-5fef-4ab2-b8ea-7b68ae085284 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:45.434427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0878345-5fef-4ab2-b8ea-7b68ae085284 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:45.434694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0878345-5fef-4ab2-b8ea-7b68ae085284 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:45.435390 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0878345-5fef-4ab2-b8ea-7b68ae085284 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:45.436769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0878345-5fef-4ab2-b8ea-7b68ae085284 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:45.448952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-d0878345-5fef-4ab2-b8ea-7b68ae085284 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 05:31:45.448952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:45.448952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:45.448952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:45.448952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:45.448952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:45.448952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:45.448952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:45.448952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 05:31:45.448952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:45.448952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:45.448952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:45.448952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:45.448952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:45.448952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:45.448952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:45.448952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:45.450202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:45.450202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:45.450202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:45.450202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:45.450202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:45.450202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:45.450202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:45.450202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:45.450202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:45.450202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:45.450202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:45.450202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:45.450202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:45.450202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:45.450202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:45.450202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:45.451317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:45.451317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:45.451317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:45.451317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:45.451317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:45.451317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:45.451317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:45.451317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:45.451317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:45.451317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:45.451317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:45.451317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:45.451317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 05:31:45.451317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:45.451317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 471/952] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jul 21 05:31:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:45.457566 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14485d4-66dc-4f28-8dea-b6dfa7c2989f None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:45.548609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:45.569253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14485d4-66dc-4f28-8dea-b6dfa7c2989f None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:45.605794 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a7186-eb9e-4f2e-b260-cd3f975fb103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:45.606576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a7186-eb9e-4f2e-b260-cd3f975fb103 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:45.606937 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a7186-eb9e-4f2e-b260-cd3f975fb103 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:45.607302 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a7186-eb9e-4f2e-b260-cd3f975fb103 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:45.608081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a7186-eb9e-4f2e-b260-cd3f975fb103 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:45.608458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a7186-eb9e-4f2e-b260-cd3f975fb103 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:45.609909 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a7186-eb9e-4f2e-b260-cd3f975fb103 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:45.615555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 482/953] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jul 21 05:31:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:45.621465 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0878345-5fef-4ab2-b8ea-7b68ae085284 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:45.680548 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:45.746504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0878345-5fef-4ab2-b8ea-7b68ae085284 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:45.781876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598fd80a-6ad6-453c-b17e-a6ed5d694b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:45.782838 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fd80a-6ad6-453c-b17e-a6ed5d694b82 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:45.783299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fd80a-6ad6-453c-b17e-a6ed5d694b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:45.783715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598fd80a-6ad6-453c-b17e-a6ed5d694b82 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:45.784564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fd80a-6ad6-453c-b17e-a6ed5d694b82 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:45.785025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fd80a-6ad6-453c-b17e-a6ed5d694b82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:45.786535 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598fd80a-6ad6-453c-b17e-a6ed5d694b82 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:45.792844 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 472/954] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jul 21 05:31:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:45.799105 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a7186-eb9e-4f2e-b260-cd3f975fb103 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:45.898138 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a7186-eb9e-4f2e-b260-cd3f975fb103 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:45.932790 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4844d7-bd65-45f9-9faf-6da8599335dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:45.933474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4844d7-bd65-45f9-9faf-6da8599335dd None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:45.933758 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4844d7-bd65-45f9-9faf-6da8599335dd None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:45.934032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4844d7-bd65-45f9-9faf-6da8599335dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:45.934791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4844d7-bd65-45f9-9faf-6da8599335dd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:45.936218 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4844d7-bd65-45f9-9faf-6da8599335dd None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:45.959257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 483/955] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jul 21 05:31:45 2022] POST /identity/v3/endpoints => generated 349 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:31:46.028642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44517cdd-5fe9-4e96-869d-601237efa32d None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:46.028929 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44517cdd-5fe9-4e96-869d-601237efa32d None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:46.029204 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44517cdd-5fe9-4e96-869d-601237efa32d None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:46.030958 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 473/956] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:46.036753 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0b5bd-de7a-4c39-8291-28a5861edf84 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:46.037020 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0b5bd-de7a-4c39-8291-28a5861edf84 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:46.037302 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0b5bd-de7a-4c39-8291-28a5861edf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:46.056091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-8ee0b5bd-de7a-4c39-8291-28a5861edf84 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:46.057218 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8ee0b5bd-de7a-4c39-8291-28a5861edf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:46.059456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:46.134102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 484/957] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:46.156034 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44517cdd-5fe9-4e96-869d-601237efa32d None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:46.266254 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44517cdd-5fe9-4e96-869d-601237efa32d None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:46.268817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:46.301498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1d09f-0024-4efd-9f72-db4d4bc183df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:46.302432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1d09f-0024-4efd-9f72-db4d4bc183df None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:46.302789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1d09f-0024-4efd-9f72-db4d4bc183df None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:46.303121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1d09f-0024-4efd-9f72-db4d4bc183df None admin] PATH_INFO: `/v3/services/compute` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:46.303794 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1d09f-0024-4efd-9f72-db4d4bc183df None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:46.305178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1d09f-0024-4efd-9f72-db4d4bc183df None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:46.317237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-31a1d09f-0024-4efd-9f72-db4d4bc183df None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 05:31:46.317237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:46.317237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:46.317237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:46.317237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:46.317237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:46.317237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:46.317237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:46.317237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 05:31:46.317237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:46.317237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:46.317237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:46.317237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:46.317237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:46.317237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:46.317237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:46.317237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:46.318346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:46.318346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:46.318346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:46.318346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:46.318346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:46.318346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:46.318346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:46.318346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:46.318346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:46.318346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:46.318346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:46.318346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:46.318346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:46.318346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:46.318346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:46.318346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:46.319338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:46.319338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:46.319338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:46.319338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:46.319338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:46.319338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:46.319338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:46.319338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:46.319338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:46.319338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:46.319338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:46.319338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:46.319338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 05:31:46.319338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:46.319338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 474/958] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:46.324650 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0b5bd-de7a-4c39-8291-28a5861edf84 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:46.435592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0b5bd-de7a-4c39-8291-28a5861edf84 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:46.471716 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cee2ea-c8e8-4efd-ac3d-741d17a5b811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:46.472399 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cee2ea-c8e8-4efd-ac3d-741d17a5b811 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:46.472684 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cee2ea-c8e8-4efd-ac3d-741d17a5b811 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:46.472946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cee2ea-c8e8-4efd-ac3d-741d17a5b811 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:46.473653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cee2ea-c8e8-4efd-ac3d-741d17a5b811 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:46.473960 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cee2ea-c8e8-4efd-ac3d-741d17a5b811 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:46.475431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cee2ea-c8e8-4efd-ac3d-741d17a5b811 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:46.476528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:46.480912 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 485/959] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jul 21 05:31:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:46.488134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1d09f-0024-4efd-9f72-db4d4bc183df None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:46.590894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1d09f-0024-4efd-9f72-db4d4bc183df None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:46.624831 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab03f3-72e7-4f41-9c67-ca8a41583cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:46.625585 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab03f3-72e7-4f41-9c67-ca8a41583cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:46.625872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab03f3-72e7-4f41-9c67-ca8a41583cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:46.626200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab03f3-72e7-4f41-9c67-ca8a41583cc1 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:46.626915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab03f3-72e7-4f41-9c67-ca8a41583cc1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:46.627254 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab03f3-72e7-4f41-9c67-ca8a41583cc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:46.628629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab03f3-72e7-4f41-9c67-ca8a41583cc1 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:46.633619 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 475/960] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jul 21 05:31:46 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:47.338452 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c5f4b-8e21-48ba-a755-c821def45a72 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:47.339458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c5f4b-8e21-48ba-a755-c821def45a72 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:47.339938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c5f4b-8e21-48ba-a755-c821def45a72 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:47.341791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 486/961] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:47.347262 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05271d-a2ed-4189-b884-614b0429252f None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:47.347764 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05271d-a2ed-4189-b884-614b0429252f None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:47.348265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05271d-a2ed-4189-b884-614b0429252f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:47.365243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686fefa-2dcb-46c0-afdb-e707dc38281a None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:47.365926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686fefa-2dcb-46c0-afdb-e707dc38281a None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:47.366422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9686fefa-2dcb-46c0-afdb-e707dc38281a None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:47.367164 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-5e05271d-a2ed-4189-b884-614b0429252f None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:47.368453 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5e05271d-a2ed-4189-b884-614b0429252f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:47.368663 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 487/962] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:47.373649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c20bb-75cb-4277-9ba6-f85616de7005 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:47.374182 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c20bb-75cb-4277-9ba6-f85616de7005 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:47.374651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c20bb-75cb-4277-9ba6-f85616de7005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:47.392390 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-df9c20bb-75cb-4277-9ba6-f85616de7005 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:47.393888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-df9c20bb-75cb-4277-9ba6-f85616de7005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:47.397145 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:47.455626 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 476/963] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:47.471091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 488/964] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:47.480230 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05271d-a2ed-4189-b884-614b0429252f None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:47.494559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c20bb-75cb-4277-9ba6-f85616de7005 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:47.602643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05271d-a2ed-4189-b884-614b0429252f None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:47.605171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c20bb-75cb-4277-9ba6-f85616de7005 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:47.631785 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:47.640631 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7be23c-861e-4d5a-800c-8512a618de31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:47.641467 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7be23c-861e-4d5a-800c-8512a618de31 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:47.641745 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7be23c-861e-4d5a-800c-8512a618de31 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:47.642015 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7be23c-861e-4d5a-800c-8512a618de31 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:47.643311 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db886ae5-823b-4c25-88bb-b29667f86312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:47.644329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db886ae5-823b-4c25-88bb-b29667f86312 None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:47.644899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db886ae5-823b-4c25-88bb-b29667f86312 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:47.645384 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db886ae5-823b-4c25-88bb-b29667f86312 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:47.646306 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db886ae5-823b-4c25-88bb-b29667f86312 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:47.648069 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db886ae5-823b-4c25-88bb-b29667f86312 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:47.649618 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: WARNING keystone.server.flask.common [None req-db886ae5-823b-4c25-88bb-b29667f86312 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 21 05:31:47.662999 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7be23c-861e-4d5a-800c-8512a618de31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:47.668622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.sql.core [None req-db886ae5-823b-4c25-88bb-b29667f86312 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 05:31:47.668622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 05:31:47.668622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [parameters: {'user_id': '80e245644b0547838247690960534a9f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 05:31:47.668622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80390) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 21 05:31:47.671791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-ff7be23c-861e-4d5a-800c-8512a618de31 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 05:31:47.671791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:47.671791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:47.671791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:47.671791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:47.671791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:47.671791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:47.671791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:47.671791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:47.671791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:47.671791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:47.671791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:47.671791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:47.671791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:47.671791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:47.671791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:47.672752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:47.672752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:47.672752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:47.672752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:47.672752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:47.672752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:47.672752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:47.672752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:47.672752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:47.672752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:47.672752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:47.672752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:47.672752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:47.672752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:47.672752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:47.672752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:47.673630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:47.673630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:47.673630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:47.673630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:47.673630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:47.673630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:47.673630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:47.673630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:47.673630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:47.673630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:47.673630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 05:31:47.673630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:47.678019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 489/965] 10.222.0.44 () {56 vars in 1120 bytes} [Thu Jul 21 05:31:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:47.678967 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7be23c-861e-4d5a-800c-8512a618de31 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:47.680610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-db886ae5-823b-4c25-88bb-b29667f86312 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 05:31:47.680610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:47.680610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 05:31:47.680610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 05:31:47.680610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 05:31:47.680610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 05:31:47.680610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 05:31:47.680610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 05:31:47.680610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 05:31:47.680610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application conn.query(q) Jul 21 05:31:47.680610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 05:31:47.680610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 05:31:47.680610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 05:31:47.680610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application result.read() Jul 21 05:31:47.680610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 05:31:47.681526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application next(self.gen) Jul 21 05:31:47.682490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 21 05:31:47.682490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application yield resource Jul 21 05:31:47.682490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 21 05:31:47.682490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application next(self.gen) Jul 21 05:31:47.682490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 21 05:31:47.682490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self.session.rollback() Jul 21 05:31:47.682490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 21 05:31:47.682490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self.force_reraise() Jul 21 05:31:47.682490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 21 05:31:47.682490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise self.value Jul 21 05:31:47.682490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 21 05:31:47.682490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 21 05:31:47.682490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 21 05:31:47.682490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application session.commit() Jul 21 05:31:47.682490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 21 05:31:47.682490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 21 05:31:47.682490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 21 05:31:47.683450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._prepare_impl() Jul 21 05:31:47.683450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 21 05:31:47.683450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self.session.flush() Jul 21 05:31:47.683450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 21 05:31:47.683450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._flush(objects) Jul 21 05:31:47.683450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 21 05:31:47.683450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 21 05:31:47.683450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 21 05:31:47.683450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application compat.raise_( Jul 21 05:31:47.683450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 05:31:47.683450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception Jul 21 05:31:47.683450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 21 05:31:47.683450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application flush_context.execute() Jul 21 05:31:47.683450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 21 05:31:47.683450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rec.execute(self) Jul 21 05:31:47.683450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 21 05:31:47.683450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 21 05:31:47.684365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 21 05:31:47.684365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 21 05:31:47.684365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 21 05:31:47.684365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 21 05:31:47.684365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 21 05:31:47.684365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 21 05:31:47.684365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 21 05:31:47.684365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 21 05:31:47.684365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 21 05:31:47.684365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 21 05:31:47.684365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 21 05:31:47.684365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 21 05:31:47.684365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 21 05:31:47.684365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 21 05:31:47.684365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 21 05:31:47.684365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception Jul 21 05:31:47.685243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 21 05:31:47.685243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 21 05:31:47.685243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 21 05:31:47.685243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 21 05:31:47.685243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 21 05:31:47.685243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application result = self._query(query) Jul 21 05:31:47.685243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 21 05:31:47.685243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application conn.query(q) Jul 21 05:31:47.685243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 21 05:31:47.685243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 21 05:31:47.685243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 21 05:31:47.685243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application result.read() Jul 21 05:31:47.685243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 21 05:31:47.685243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 21 05:31:47.685243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 21 05:31:47.685243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 21 05:31:47.686094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 21 05:31:47.686094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 21 05:31:47.686094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 21 05:31:47.686094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 21 05:31:47.686094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 21 05:31:47.686094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 21 05:31:47.686094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [parameters: {'user_id': '80e245644b0547838247690960534a9f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 21 05:31:47.686094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 21 05:31:47.686094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:47.686094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:47.686094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:47.686094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:47.686094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:47.686094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:47.686094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:47.686894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:47.686894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:47.686894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:47.686894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:47.686894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:47.686894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:47.686894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:47.686894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 21 05:31:47.686894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 21 05:31:47.686894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:47.686894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:47.686894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:47.686894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:47.686894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:47.686894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:47.686894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 21 05:31:47.686894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 21 05:31:47.690036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 21 05:31:47.690036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 21 05:31:47.690036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 21 05:31:47.690036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 21 05:31:47.690036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 21 05:31:47.690036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:47.697554 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 477/966] 10.222.0.44 () {60 vars in 1131 bytes} [Thu Jul 21 05:31:47 2022] POST /identity/v3/users => generated 160 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:47.699114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db886ae5-823b-4c25-88bb-b29667f86312 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:47.728845 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:47.786705 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7be23c-861e-4d5a-800c-8512a618de31 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:47.815584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db886ae5-823b-4c25-88bb-b29667f86312 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:47.822387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde4b16-7539-414a-ab10-26907a832c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:47.823095 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde4b16-7539-414a-ab10-26907a832c05 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:47.823379 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde4b16-7539-414a-ab10-26907a832c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:47.823651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde4b16-7539-414a-ab10-26907a832c05 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:47.824396 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde4b16-7539-414a-ab10-26907a832c05 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:47.824700 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde4b16-7539-414a-ab10-26907a832c05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:47.826292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde4b16-7539-414a-ab10-26907a832c05 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:47.839960 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 490/967] 10.222.0.44 () {56 vars in 1115 bytes} [Thu Jul 21 05:31:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:47.840928 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde4b16-7539-414a-ab10-26907a832c05 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:47.854823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f25fa6-50e1-4624-9e66-034c7965f90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:47.855961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f25fa6-50e1-4624-9e66-034c7965f90c None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:47.856553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f25fa6-50e1-4624-9e66-034c7965f90c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:47.857022 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f25fa6-50e1-4624-9e66-034c7965f90c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:47.865435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f25fa6-50e1-4624-9e66-034c7965f90c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:47.867613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f25fa6-50e1-4624-9e66-034c7965f90c None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:47.883413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-00f25fa6-50e1-4624-9e66-034c7965f90c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 05:31:47.883413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:47.883413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:47.883413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:47.883413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:47.883413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:47.883413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:47.883413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:47.883413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:47.883413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:47.883413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:47.883413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:47.883413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:47.883413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:47.883413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:47.883413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:47.884409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:47.885407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:47.885407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:47.885407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:47.885407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:47.885407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:47.885407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:47.885407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:47.885407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:47.885407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 21 05:31:47.885407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:47.892526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 478/968] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:31:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:47.893953 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:47.895518 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f25fa6-50e1-4624-9e66-034c7965f90c None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:47.944818 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde4b16-7539-414a-ab10-26907a832c05 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:47.979531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927b89f-e77f-4fdd-85f4-f75654f39f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:47.980501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927b89f-e77f-4fdd-85f4-f75654f39f7a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:47.980719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927b89f-e77f-4fdd-85f4-f75654f39f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:47.980985 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927b89f-e77f-4fdd-85f4-f75654f39f7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:47.988886 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:47.993393 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927b89f-e77f-4fdd-85f4-f75654f39f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:47.994942 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927b89f-e77f-4fdd-85f4-f75654f39f7a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:48.006473 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f25fa6-50e1-4624-9e66-034c7965f90c None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:48.015219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-3927b89f-e77f-4fdd-85f4-f75654f39f7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:48.015219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:48.015219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:48.015219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:48.015219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:48.015219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:48.015219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:48.015219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:48.015219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:48.015219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:48.015219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:48.015219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:48.015219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:48.015219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:48.015219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:48.016127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:48.016127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:48.016127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:48.016127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:48.016127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:48.016127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:48.016127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:48.016127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:48.016127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:48.016127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:48.016127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:48.016127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:48.016127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:48.016127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:48.016127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:48.016127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:48.016127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:48.017026 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:48.017026 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:48.017026 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:48.017026 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:48.021171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 491/969] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:48.022289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927b89f-e77f-4fdd-85f4-f75654f39f7a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:48.046644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e74149-85ee-4378-9c94-b1cc43507654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:48.047725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e74149-85ee-4378-9c94-b1cc43507654 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:48.048318 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e74149-85ee-4378-9c94-b1cc43507654 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:48.048805 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e74149-85ee-4378-9c94-b1cc43507654 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:48.049899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e74149-85ee-4378-9c94-b1cc43507654 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:48.050421 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e74149-85ee-4378-9c94-b1cc43507654 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:48.052326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e74149-85ee-4378-9c94-b1cc43507654 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:48.094705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 479/970] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jul 21 05:31:47 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:48.111152 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e86e0d-de63-4df8-a1e0-5e70e51a56f0 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:48.111152 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e86e0d-de63-4df8-a1e0-5e70e51a56f0 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:48.111152 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e86e0d-de63-4df8-a1e0-5e70e51a56f0 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:48.111152 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 480/971] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:48.111152 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0aae8-2dc1-4219-b61a-bb1fa0768be1 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:48.111152 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0aae8-2dc1-4219-b61a-bb1fa0768be1 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:48.111819 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0aae8-2dc1-4219-b61a-bb1fa0768be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:48.129350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-61e0aae8-2dc1-4219-b61a-bb1fa0768be1 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:48.130966 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-61e0aae8-2dc1-4219-b61a-bb1fa0768be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:48.142231 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927b89f-e77f-4fdd-85f4-f75654f39f7a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:48.144257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:48.185388 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a085c57-e470-499d-936e-c650ec3105af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:48.186234 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a085c57-e470-499d-936e-c650ec3105af None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:48.186654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a085c57-e470-499d-936e-c650ec3105af None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:48.187052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a085c57-e470-499d-936e-c650ec3105af None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:48.187905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a085c57-e470-499d-936e-c650ec3105af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:48.188319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a085c57-e470-499d-936e-c650ec3105af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:48.189854 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a085c57-e470-499d-936e-c650ec3105af None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:48.203474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 492/972] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:48.209443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a085c57-e470-499d-936e-c650ec3105af None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:48.212769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 481/973] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:48.235830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0aae8-2dc1-4219-b61a-bb1fa0768be1 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:48.331405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:48.346868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a085c57-e470-499d-936e-c650ec3105af None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:48.368633 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0aae8-2dc1-4219-b61a-bb1fa0768be1 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:48.396826 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f4118-4a75-4ab4-b411-238c73c2378b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:48.399703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f4118-4a75-4ab4-b411-238c73c2378b None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:48.399703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f4118-4a75-4ab4-b411-238c73c2378b None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:48.399703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f4118-4a75-4ab4-b411-238c73c2378b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:48.408877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f4118-4a75-4ab4-b411-238c73c2378b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:48.410574 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f4118-4a75-4ab4-b411-238c73c2378b None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:48.416220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf315d0-ae85-448a-b246-1da368165521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:48.417046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf315d0-ae85-448a-b246-1da368165521 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:48.417325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf315d0-ae85-448a-b246-1da368165521 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:48.417596 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf315d0-ae85-448a-b246-1da368165521 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:48.418255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf315d0-ae85-448a-b246-1da368165521 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:48.419660 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf315d0-ae85-448a-b246-1da368165521 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:48.432326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-603f4118-4a75-4ab4-b411-238c73c2378b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 05:31:48.432326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:48.432326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:48.432326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:48.432326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:48.432326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:48.432326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:48.432326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:48.432326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:48.432326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:48.432326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:48.432326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:48.432326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:48.432326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:48.432326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:48.432326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:48.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:48.434437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:48.434437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:48.434437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:48.434437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:48.434437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:48.434437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:48.434437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:48.434437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:48.434437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 21 05:31:48.434437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:48.435274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-2cf315d0-ae85-448a-b246-1da368165521 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 05:31:48.435274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:48.435274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:48.435274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:48.435274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:48.435274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:48.435274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:48.435274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:48.435274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 05:31:48.435274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:48.435274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:48.435274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:48.435274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:48.435274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:48.435274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:48.435274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:48.435274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:48.436445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:48.436445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:48.436445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:48.436445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:48.436445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:48.436445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:48.436445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:48.436445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:48.436445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:48.436445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:48.436445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:48.436445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:48.436445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:48.436445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:48.436445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:48.436445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:48.437309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:48.437309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:48.437309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:48.437309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:48.437309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:48.437309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:48.437309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:48.437309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:48.437309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:48.437309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:48.437309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:48.437309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:48.437309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 05:31:48.437309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:48.438063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 493/974] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jul 21 05:31:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:48.438134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 482/975] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jul 21 05:31:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:48.440665 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f4118-4a75-4ab4-b411-238c73c2378b None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:48.442408 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf315d0-ae85-448a-b246-1da368165521 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:48.467575 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:48.544445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f4118-4a75-4ab4-b411-238c73c2378b None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:48.551117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf315d0-ae85-448a-b246-1da368165521 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:48.579555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b65dd3-e996-4efc-be82-a88d1a99ac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:48.580247 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b65dd3-e996-4efc-be82-a88d1a99ac13 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:48.580520 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b65dd3-e996-4efc-be82-a88d1a99ac13 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:48.580791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b65dd3-e996-4efc-be82-a88d1a99ac13 None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:48.581622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b65dd3-e996-4efc-be82-a88d1a99ac13 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:48.581921 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b65dd3-e996-4efc-be82-a88d1a99ac13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:48.583374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b65dd3-e996-4efc-be82-a88d1a99ac13 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:48.589022 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a67849-d304-4fd4-a2a8-5e4493cedbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:48.589811 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a67849-d304-4fd4-a2a8-5e4493cedbfd None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:48.590181 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a67849-d304-4fd4-a2a8-5e4493cedbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:48.590523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a67849-d304-4fd4-a2a8-5e4493cedbfd None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:48.591341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a67849-d304-4fd4-a2a8-5e4493cedbfd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:48.591723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a67849-d304-4fd4-a2a8-5e4493cedbfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:48.593158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a67849-d304-4fd4-a2a8-5e4493cedbfd None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:48.595432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:48.599055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 483/976] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jul 21 05:31:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:48.606090 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 494/977] 10.222.0.44 () {56 vars in 1149 bytes} [Thu Jul 21 05:31:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:48.606246 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a67849-d304-4fd4-a2a8-5e4493cedbfd None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:48.607223 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b65dd3-e996-4efc-be82-a88d1a99ac13 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:48.693878 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:48.714573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b65dd3-e996-4efc-be82-a88d1a99ac13 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:48.717123 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a67849-d304-4fd4-a2a8-5e4493cedbfd None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:48.747706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b7e41-7b08-4488-91bb-0f5472c622d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:48.748355 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b7e41-7b08-4488-91bb-0f5472c622d5 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:48.748636 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b7e41-7b08-4488-91bb-0f5472c622d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:48.748914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b7e41-7b08-4488-91bb-0f5472c622d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:48.750199 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c047fa8-336a-4621-a91a-5f054af335a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:48.750867 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c047fa8-336a-4621-a91a-5f054af335a2 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:48.751187 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c047fa8-336a-4621-a91a-5f054af335a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:48.751466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c047fa8-336a-4621-a91a-5f054af335a2 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:48.752192 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c047fa8-336a-4621-a91a-5f054af335a2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:48.752496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c047fa8-336a-4621-a91a-5f054af335a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:48.753870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c047fa8-336a-4621-a91a-5f054af335a2 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:48.759205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 484/978] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jul 21 05:31:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:48.760484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e7d27-bd08-47ac-a2c8-bf253d470e73 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:48.760788 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e7d27-bd08-47ac-a2c8-bf253d470e73 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:48.761064 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534e7d27-bd08-47ac-a2c8-bf253d470e73 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:48.763179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 485/979] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:48.767235 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e7d27-bd08-47ac-a2c8-bf253d470e73 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:48.816270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b7e41-7b08-4488-91bb-0f5472c622d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:48.817835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b7e41-7b08-4488-91bb-0f5472c622d5 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:48.831170 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:48.869546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534e7d27-bd08-47ac-a2c8-bf253d470e73 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:48.894282 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:48.903520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a491f8b-6e69-4da7-8c3a-60bcc3783d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:48.904179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a491f8b-6e69-4da7-8c3a-60bcc3783d09 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:48.904458 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a491f8b-6e69-4da7-8c3a-60bcc3783d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:48.904721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a491f8b-6e69-4da7-8c3a-60bcc3783d09 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:48.905527 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a491f8b-6e69-4da7-8c3a-60bcc3783d09 None admin] RBAC: Adding query filter params (interface=admin, service_id=3f47157483f44de98b041a1f4e12f212, region_id=RegionOne) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:48.905838 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a491f8b-6e69-4da7-8c3a-60bcc3783d09 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:48.907222 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a491f8b-6e69-4da7-8c3a-60bcc3783d09 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:48.912876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 486/980] 10.222.0.44 () {56 vars in 1249 bytes} [Thu Jul 21 05:31:48 2022] GET /identity/v3/endpoints?service_id=3f47157483f44de98b041a1f4e12f212&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:48.913973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f57c1b-4a9f-4ad3-972a-d5ae88c05d8f None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:48.914273 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f57c1b-4a9f-4ad3-972a-d5ae88c05d8f None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:48.914528 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f57c1b-4a9f-4ad3-972a-d5ae88c05d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:48.919561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 495/981] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:48 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 05:31:48.923901 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b7e41-7b08-4488-91bb-0f5472c622d5 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:48.930694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-32f57c1b-4a9f-4ad3-972a-d5ae88c05d8f None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:48.931902 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-32f57c1b-4a9f-4ad3-972a-d5ae88c05d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:48.998147 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 487/982] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:48.998974 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f57c1b-4a9f-4ad3-972a-d5ae88c05d8f None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:49.022061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b7e41-7b08-4488-91bb-0f5472c622d5 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:49.055792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2ce6c-0a16-4417-a880-939c45a5bf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:49.056599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2ce6c-0a16-4417-a880-939c45a5bf03 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:49.056981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2ce6c-0a16-4417-a880-939c45a5bf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:49.057355 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2ce6c-0a16-4417-a880-939c45a5bf03 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:49.058135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2ce6c-0a16-4417-a880-939c45a5bf03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:49.059655 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2ce6c-0a16-4417-a880-939c45a5bf03 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:49.060805 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:49.065798 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 496/983] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jul 21 05:31:48 2022] GET /identity/v3/services => generated 1497 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:31:49.066628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2ce6c-0a16-4417-a880-939c45a5bf03 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:49.107530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f57c1b-4a9f-4ad3-972a-d5ae88c05d8f None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:49.120691 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:49.149844 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729bda9-9ef9-409c-b4a8-9018e396477f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:49.150785 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729bda9-9ef9-409c-b4a8-9018e396477f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:49.151111 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729bda9-9ef9-409c-b4a8-9018e396477f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:49.151397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729bda9-9ef9-409c-b4a8-9018e396477f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:49.164253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729bda9-9ef9-409c-b4a8-9018e396477f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:49.165675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729bda9-9ef9-409c-b4a8-9018e396477f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:49.184342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2ce6c-0a16-4417-a880-939c45a5bf03 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:49.185988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-d729bda9-9ef9-409c-b4a8-9018e396477f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:49.185988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:49.185988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:49.185988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:49.185988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:49.185988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:49.185988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:49.185988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:49.185988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:49.185988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:49.185988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:49.185988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:49.185988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:49.185988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:49.185988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:49.187086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:49.187086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:49.187086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:49.187086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:49.187086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:49.187086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:49.187086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:49.187086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:49.187086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:49.187086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:49.187086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:49.187086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:49.187086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:49.187086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:49.187086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:49.187086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:49.187086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:49.188171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:49.188171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:49.188171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:49.188171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:49.194771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 488/984] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:49.194985 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729bda9-9ef9-409c-b4a8-9018e396477f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:49.234405 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74edce-fcdd-4db3-bc2c-2d452cfb5703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:49.236169 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74edce-fcdd-4db3-bc2c-2d452cfb5703 None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:49.236619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74edce-fcdd-4db3-bc2c-2d452cfb5703 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:49.237030 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74edce-fcdd-4db3-bc2c-2d452cfb5703 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:49.237936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74edce-fcdd-4db3-bc2c-2d452cfb5703 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:49.239548 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74edce-fcdd-4db3-bc2c-2d452cfb5703 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:49.256424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 497/985] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jul 21 05:31:49 2022] POST /identity/v3/services => generated 242 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:31:49.311735 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729bda9-9ef9-409c-b4a8-9018e396477f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:49.325207 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:49.361685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143b722-da0d-4504-a428-3b1ce7c29238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:49.362417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143b722-da0d-4504-a428-3b1ce7c29238 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:49.362702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143b722-da0d-4504-a428-3b1ce7c29238 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:49.362974 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143b722-da0d-4504-a428-3b1ce7c29238 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:49.363801 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143b722-da0d-4504-a428-3b1ce7c29238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:49.364096 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143b722-da0d-4504-a428-3b1ce7c29238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:49.365630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143b722-da0d-4504-a428-3b1ce7c29238 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:49.380227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 489/986] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:49.385517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74edce-fcdd-4db3-bc2c-2d452cfb5703 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:49.492506 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74edce-fcdd-4db3-bc2c-2d452cfb5703 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:49.494749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:49.527391 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35866d10-1279-487b-8caf-f832b96a514e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:49.528201 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35866d10-1279-487b-8caf-f832b96a514e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:49.528475 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35866d10-1279-487b-8caf-f832b96a514e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:49.528749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35866d10-1279-487b-8caf-f832b96a514e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:49.535652 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35866d10-1279-487b-8caf-f832b96a514e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:49.537097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35866d10-1279-487b-8caf-f832b96a514e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:49.551910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-35866d10-1279-487b-8caf-f832b96a514e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:49.551910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:49.551910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:49.551910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:49.551910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:49.551910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:49.551910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:49.551910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:49.551910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:49.551910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:49.551910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:49.551910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:49.551910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:49.551910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:49.551910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:49.552868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:49.552868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:49.552868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:49.552868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:49.552868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:49.552868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:49.552868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:49.552868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:49.552868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:49.552868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:49.552868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:49.552868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:49.552868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:49.552868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:49.552868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:49.552868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:49.552868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:49.553895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:49.553895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:49.553895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:49.553895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:49.553895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:49.553895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:49.553895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:49.553895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:49.553895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 498/987] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:49.559368 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143b722-da0d-4504-a428-3b1ce7c29238 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:49.670798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143b722-da0d-4504-a428-3b1ce7c29238 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:49.704389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494061a-6323-472b-8845-2e4aeb469829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:49.705062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494061a-6323-472b-8845-2e4aeb469829 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:49.705343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494061a-6323-472b-8845-2e4aeb469829 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:49.705616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494061a-6323-472b-8845-2e4aeb469829 None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:49.706402 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494061a-6323-472b-8845-2e4aeb469829 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:49.706707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494061a-6323-472b-8845-2e4aeb469829 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:49.708174 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494061a-6323-472b-8845-2e4aeb469829 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:49.725769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 490/988] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:49.726809 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35866d10-1279-487b-8caf-f832b96a514e None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:49.830414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35866d10-1279-487b-8caf-f832b96a514e None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:49.865123 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fcd38-25e2-49ce-9a57-e9dda69be8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:49.865837 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fcd38-25e2-49ce-9a57-e9dda69be8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:49.866112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fcd38-25e2-49ce-9a57-e9dda69be8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:49.866385 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fcd38-25e2-49ce-9a57-e9dda69be8e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:49.867236 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66fcd38-25e2-49ce-9a57-e9dda69be8e6 None admin] RBAC: Adding query filter params (role.id=545b77cdab164cd29c9b639b341c4018, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=4d04998b636649d98103b1cd5f3a0f7c) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:49.867540 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66fcd38-25e2-49ce-9a57-e9dda69be8e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:49.868972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66fcd38-25e2-49ce-9a57-e9dda69be8e6 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:49.874873 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 499/989] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:49 2022] GET /identity/v3/role_assignments?user.id=4d04998b636649d98103b1cd5f3a0f7c&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=545b77cdab164cd29c9b639b341c4018 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:51.125043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73f29b-8247-4c0b-9802-0df6ed83cb66 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:51.125563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73f29b-8247-4c0b-9802-0df6ed83cb66 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:51.125563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73f29b-8247-4c0b-9802-0df6ed83cb66 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:51.126838 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 491/990] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:51.132782 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac2347-602d-48a9-b91d-d4f110c0a1cf None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:51.133216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac2347-602d-48a9-b91d-d4f110c0a1cf None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:51.133408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac2347-602d-48a9-b91d-d4f110c0a1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:51.150951 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-aeac2347-602d-48a9-b91d-d4f110c0a1cf None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:51.152152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-aeac2347-602d-48a9-b91d-d4f110c0a1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:51.177554 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:51.229290 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 500/991] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:51.252406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73f29b-8247-4c0b-9802-0df6ed83cb66 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:51.280541 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:51.310070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e00254-b645-4da8-89e9-9bb9aa3556b3 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:51.310281 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e00254-b645-4da8-89e9-9bb9aa3556b3 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:51.310500 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e00254-b645-4da8-89e9-9bb9aa3556b3 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:51.316206 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 501/992] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:31:51.317650 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fbe96-cf8b-4766-9a00-9e597956dee2 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:51.317819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fbe96-cf8b-4766-9a00-9e597956dee2 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:51.318100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800fbe96-cf8b-4766-9a00-9e597956dee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:51.334681 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-800fbe96-cf8b-4766-9a00-9e597956dee2 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:51.335917 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-800fbe96-cf8b-4766-9a00-9e597956dee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:51.369641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73f29b-8247-4c0b-9802-0df6ed83cb66 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:51.411207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 502/993] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:51.412765 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322ef8c-5ce0-49dd-bfe2-2a735be225f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:51.413863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322ef8c-5ce0-49dd-bfe2-2a735be225f2 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:51.414356 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322ef8c-5ce0-49dd-bfe2-2a735be225f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:51.414823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322ef8c-5ce0-49dd-bfe2-2a735be225f2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:51.415735 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322ef8c-5ce0-49dd-bfe2-2a735be225f2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:51.417360 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322ef8c-5ce0-49dd-bfe2-2a735be225f2 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:51.429083 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fbe96-cf8b-4766-9a00-9e597956dee2 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:51.431830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-4322ef8c-5ce0-49dd-bfe2-2a735be225f2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 05:31:51.431830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:51.431830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:51.431830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:51.431830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:51.431830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:51.431830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:51.431830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:51.431830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 05:31:51.431830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:51.431830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:51.431830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:51.431830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:51.431830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:51.431830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:51.431830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:51.431830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:51.432847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:51.432847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:51.432847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:51.432847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:51.432847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:51.432847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:51.432847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:51.432847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:51.432847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:51.432847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:51.432847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:51.432847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:51.432847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:51.432847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:51.432847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:51.432847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:51.433704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:51.433704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:51.433704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:51.433704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:51.433704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:51.433704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:51.433704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:51.433704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:51.433704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:51.433704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:51.433704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:51.433704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:51.433704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 21 05:31:51.433704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:51.436416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 492/994] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jul 21 05:31:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:51.444611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322ef8c-5ce0-49dd-bfe2-2a735be225f2 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:51.543105 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:51.547464 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:51.557601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800fbe96-cf8b-4766-9a00-9e597956dee2 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:51.559291 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322ef8c-5ce0-49dd-bfe2-2a735be225f2 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:51.593942 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f487d6a-2a7d-4290-81d0-b1a50975ccde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:51.595135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f487d6a-2a7d-4290-81d0-b1a50975ccde None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:51.595613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f487d6a-2a7d-4290-81d0-b1a50975ccde None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:51.596059 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f487d6a-2a7d-4290-81d0-b1a50975ccde None admin] PATH_INFO: `/v3/services/compute` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:51.596884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f487d6a-2a7d-4290-81d0-b1a50975ccde None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:51.598549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f487d6a-2a7d-4290-81d0-b1a50975ccde None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:51.601215 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d9311-5052-4e4f-9985-4a929a5f4a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:51.602161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d9311-5052-4e4f-9985-4a929a5f4a36 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:51.602450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d9311-5052-4e4f-9985-4a929a5f4a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:51.602722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d9311-5052-4e4f-9985-4a929a5f4a36 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:51.603719 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d9311-5052-4e4f-9985-4a929a5f4a36 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:51.604032 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d9311-5052-4e4f-9985-4a929a5f4a36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:51.605401 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d9311-5052-4e4f-9985-4a929a5f4a36 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:51.610744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 493/995] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jul 21 05:31:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:51.612776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-7f487d6a-2a7d-4290-81d0-b1a50975ccde None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 05:31:51.612776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:51.612776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:51.612776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:51.612776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:51.612776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:51.612776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:51.612776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:51.612776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 05:31:51.612776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:51.612776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:51.612776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:51.612776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:51.612776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:51.612776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:51.612776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:51.612776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:51.613776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:51.613776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:51.613776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:51.613776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:51.613776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:51.613776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:51.613776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:51.613776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:51.613776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:51.613776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:51.613776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:51.613776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:51.613776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:51.613776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:51.613776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:51.613776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:51.614626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:51.614626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:51.614626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:51.614626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:51.614626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:51.614626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:51.614626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:51.614626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:51.614626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:51.614626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:51.614626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:51.614626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:51.614626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 05:31:51.614626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:51.622033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 503/996] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:51.622185 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d9311-5052-4e4f-9985-4a929a5f4a36 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:51.623360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f487d6a-2a7d-4290-81d0-b1a50975ccde None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:51.731729 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f487d6a-2a7d-4290-81d0-b1a50975ccde None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:51.738320 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d9311-5052-4e4f-9985-4a929a5f4a36 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:51.768372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b82c4-a4ad-4297-8c1d-90fba10c4108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:51.769094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b82c4-a4ad-4297-8c1d-90fba10c4108 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:51.769348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b82c4-a4ad-4297-8c1d-90fba10c4108 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:51.769623 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b82c4-a4ad-4297-8c1d-90fba10c4108 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:51.770529 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b82c4-a4ad-4297-8c1d-90fba10c4108 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:51.770842 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b82c4-a4ad-4297-8c1d-90fba10c4108 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:51.772480 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b82c4-a4ad-4297-8c1d-90fba10c4108 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:51.772922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53286ab-5580-4227-afe3-c0d0520c776c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:51.773595 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53286ab-5580-4227-afe3-c0d0520c776c None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:51.773876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53286ab-5580-4227-afe3-c0d0520c776c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:51.774143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53286ab-5580-4227-afe3-c0d0520c776c None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:51.774836 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53286ab-5580-4227-afe3-c0d0520c776c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:51.775175 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53286ab-5580-4227-afe3-c0d0520c776c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:51.776529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53286ab-5580-4227-afe3-c0d0520c776c None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:51.777517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 504/997] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jul 21 05:31:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:51.781849 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 494/998] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jul 21 05:31:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:51.783367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b82c4-a4ad-4297-8c1d-90fba10c4108 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:51.788559 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53286ab-5580-4227-afe3-c0d0520c776c None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:51.802707 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:51.811131 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:51.884134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b82c4-a4ad-4297-8c1d-90fba10c4108 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:51.902385 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53286ab-5580-4227-afe3-c0d0520c776c None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:51.916746 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73f9cc-82d2-40f8-844f-55787258094a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:51.917355 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73f9cc-82d2-40f8-844f-55787258094a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:51.917635 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73f9cc-82d2-40f8-844f-55787258094a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:51.917979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73f9cc-82d2-40f8-844f-55787258094a None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:51.918687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73f9cc-82d2-40f8-844f-55787258094a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:51.918997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73f9cc-82d2-40f8-844f-55787258094a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:51.920390 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73f9cc-82d2-40f8-844f-55787258094a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:51.925259 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 505/999] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jul 21 05:31:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:51.931120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73f9cc-82d2-40f8-844f-55787258094a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:51.938851 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3f219-dad3-405e-ab3d-8e3ee4a77c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:51.939556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3f219-dad3-405e-ab3d-8e3ee4a77c9b None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:51.939851 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3f219-dad3-405e-ab3d-8e3ee4a77c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:51.940161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3f219-dad3-405e-ab3d-8e3ee4a77c9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:51.940924 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3f219-dad3-405e-ab3d-8e3ee4a77c9b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:51.942305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3f219-dad3-405e-ab3d-8e3ee4a77c9b None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:51.959756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 495/1000] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jul 21 05:31:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:31:51.961087 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fea327-c95c-4955-b4f5-e4ebbc3b6d35 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:51.961371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fea327-c95c-4955-b4f5-e4ebbc3b6d35 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:51.961670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fea327-c95c-4955-b4f5-e4ebbc3b6d35 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:51.962927 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 496/1001] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:51.976017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0a935-4a1d-4915-b17c-46a0dedabf92 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:51.976358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0a935-4a1d-4915-b17c-46a0dedabf92 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:51.976775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0a935-4a1d-4915-b17c-46a0dedabf92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:51.995419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-27a0a935-4a1d-4915-b17c-46a0dedabf92 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:51.997117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-27a0a935-4a1d-4915-b17c-46a0dedabf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:52.018795 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:52.041879 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73f9cc-82d2-40f8-844f-55787258094a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:52.084892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d119b08-875c-4380-83f8-bb658505ea79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:52.085571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d119b08-875c-4380-83f8-bb658505ea79 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:52.085854 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d119b08-875c-4380-83f8-bb658505ea79 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:52.086122 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d119b08-875c-4380-83f8-bb658505ea79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:52.091330 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d119b08-875c-4380-83f8-bb658505ea79 None admin] RBAC: Adding query filter params (interface=public, service_id=63bc87d4a23e47b1852a69e7c63e149d, region_id=RegionOne) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:52.091540 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 497/1002] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:51 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:52.091636 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d119b08-875c-4380-83f8-bb658505ea79 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:52.091636 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d119b08-875c-4380-83f8-bb658505ea79 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:52.099961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 506/1003] 10.222.0.44 () {56 vars in 1251 bytes} [Thu Jul 21 05:31:51 2022] GET /identity/v3/endpoints?service_id=63bc87d4a23e47b1852a69e7c63e149d&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:52.101329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0a935-4a1d-4915-b17c-46a0dedabf92 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:52.110346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:52.110724 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d119b08-875c-4380-83f8-bb658505ea79 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:52.214638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0a935-4a1d-4915-b17c-46a0dedabf92 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:52.219195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d119b08-875c-4380-83f8-bb658505ea79 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:52.253502 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72abbff-4ac2-483d-a640-9960dfae4b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:52.254058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca78bbd-d2e6-4897-b818-04ffa725c138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:52.254294 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72abbff-4ac2-483d-a640-9960dfae4b8c None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:52.254574 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72abbff-4ac2-483d-a640-9960dfae4b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:52.254734 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca78bbd-d2e6-4897-b818-04ffa725c138 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:52.254831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72abbff-4ac2-483d-a640-9960dfae4b8c None admin] PATH_INFO: `/v3/services/image` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:52.255051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca78bbd-d2e6-4897-b818-04ffa725c138 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:52.255325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca78bbd-d2e6-4897-b818-04ffa725c138 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:52.255518 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72abbff-4ac2-483d-a640-9960dfae4b8c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:52.256015 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca78bbd-d2e6-4897-b818-04ffa725c138 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:52.257045 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72abbff-4ac2-483d-a640-9960dfae4b8c None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:52.257499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca78bbd-d2e6-4897-b818-04ffa725c138 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:52.270774 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 498/1004] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jul 21 05:31:52 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:31:52.270983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-d72abbff-4ac2-483d-a640-9960dfae4b8c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 05:31:52.270983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:52.270983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:52.270983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:52.270983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:52.270983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:52.270983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:52.270983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:52.270983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 05:31:52.270983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:52.270983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:52.270983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:52.270983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:52.270983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:52.270983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:52.270983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:52.270983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:52.272374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:52.272374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:52.272374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:52.272374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:52.272374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:52.272374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:52.272374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:52.272374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:52.272374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:52.272374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:52.272374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:52.272374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:52.272374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:52.272374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:52.272374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:52.272374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:52.273453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:52.273453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:52.273453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:52.273453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:52.273453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:52.273453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:52.273453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:52.273453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:52.273453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:52.273453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:52.273453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:52.273453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:52.273453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 05:31:52.273453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:52.273453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 507/1005] 10.222.0.44 () {56 vars in 1105 bytes} [Thu Jul 21 05:31:52 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:52.276366 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca78bbd-d2e6-4897-b818-04ffa725c138 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:52.352339 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:52.388607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca78bbd-d2e6-4897-b818-04ffa725c138 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:52.425591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168cd1f-2203-4f9a-8c80-86f970f78995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:52.426272 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168cd1f-2203-4f9a-8c80-86f970f78995 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:52.426551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168cd1f-2203-4f9a-8c80-86f970f78995 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:52.426824 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168cd1f-2203-4f9a-8c80-86f970f78995 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:52.427594 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168cd1f-2203-4f9a-8c80-86f970f78995 None admin] RBAC: Adding query filter params (name=image) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:52.428009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168cd1f-2203-4f9a-8c80-86f970f78995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:52.429390 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168cd1f-2203-4f9a-8c80-86f970f78995 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:52.436781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 499/1006] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:52.442476 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72abbff-4ac2-483d-a640-9960dfae4b8c None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:52.447290 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:52.545158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72abbff-4ac2-483d-a640-9960dfae4b8c None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:52.577959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26903e7c-68d3-4ee1-b59a-d6cab8eea6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:52.578626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26903e7c-68d3-4ee1-b59a-d6cab8eea6de None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:52.578903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26903e7c-68d3-4ee1-b59a-d6cab8eea6de None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:52.579202 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26903e7c-68d3-4ee1-b59a-d6cab8eea6de None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:52.579902 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26903e7c-68d3-4ee1-b59a-d6cab8eea6de None admin] RBAC: Adding query filter params (type=image) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:52.580205 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26903e7c-68d3-4ee1-b59a-d6cab8eea6de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:52.581573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26903e7c-68d3-4ee1-b59a-d6cab8eea6de None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:52.586431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 508/1007] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:52 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:54.018901 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdf343-7bd3-444f-a82b-4bb1a3533685 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:54.019519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdf343-7bd3-444f-a82b-4bb1a3533685 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:54.019519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdf343-7bd3-444f-a82b-4bb1a3533685 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:54.020953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 500/1008] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:54.026528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3115b-f3c8-48a5-82b1-fc87c152f7d9 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:54.026970 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3115b-f3c8-48a5-82b1-fc87c152f7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:54.027167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3115b-f3c8-48a5-82b1-fc87c152f7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:54.045406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-b9f3115b-f3c8-48a5-82b1-fc87c152f7d9 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:54.046613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b9f3115b-f3c8-48a5-82b1-fc87c152f7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:54.111402 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:54.123172 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 509/1009] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:54 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:54.141517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdf343-7bd3-444f-a82b-4bb1a3533685 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:54.249513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdf343-7bd3-444f-a82b-4bb1a3533685 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:54.280382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:54.286616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a67be4-1f30-4213-a95b-a81e6cd75810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:54.287824 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a67be4-1f30-4213-a95b-a81e6cd75810 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:54.288109 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a67be4-1f30-4213-a95b-a81e6cd75810 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:54.288379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a67be4-1f30-4213-a95b-a81e6cd75810 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:54.289864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a67be4-1f30-4213-a95b-a81e6cd75810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:54.290447 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1e948-d137-4aac-a058-cfad33a4375c None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:54.290957 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1e948-d137-4aac-a058-cfad33a4375c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:54.291361 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a67be4-1f30-4213-a95b-a81e6cd75810 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:54.291739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1e948-d137-4aac-a058-cfad33a4375c None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:54.293035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 501/1010] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:54.297761 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 510/1011] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:54.298756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a67be4-1f30-4213-a95b-a81e6cd75810 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:54.299355 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040ff1-e0c1-4737-afdf-d42261283cd7 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:54.299628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040ff1-e0c1-4737-afdf-d42261283cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:54.299913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040ff1-e0c1-4737-afdf-d42261283cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:54.316187 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-68040ff1-e0c1-4737-afdf-d42261283cd7 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:54.317373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-68040ff1-e0c1-4737-afdf-d42261283cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:54.389374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 511/1012] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:54 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:54.411278 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a67be4-1f30-4213-a95b-a81e6cd75810 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:54.411858 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68040ff1-e0c1-4737-afdf-d42261283cd7 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:54.448381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd0d41-1765-40a6-9c2e-1a170b120e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:54.449052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd0d41-1765-40a6-9c2e-1a170b120e2a None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:54.449336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd0d41-1765-40a6-9c2e-1a170b120e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:54.449613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd0d41-1765-40a6-9c2e-1a170b120e2a None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:54.450338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd0d41-1765-40a6-9c2e-1a170b120e2a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:54.451807 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd0d41-1765-40a6-9c2e-1a170b120e2a None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:54.478415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 502/1013] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jul 21 05:31:54 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:31:54.529427 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68040ff1-e0c1-4737-afdf-d42261283cd7 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:54.560426 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:54.570570 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1aeca-faf9-4555-ab2d-9c06503af2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:54.571425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1aeca-faf9-4555-ab2d-9c06503af2fb None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:54.571704 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1aeca-faf9-4555-ab2d-9c06503af2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:54.571970 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1aeca-faf9-4555-ab2d-9c06503af2fb None admin] PATH_INFO: `/v3/services/compute` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:54.572653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1aeca-faf9-4555-ab2d-9c06503af2fb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:54.574092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1aeca-faf9-4555-ab2d-9c06503af2fb None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:54.586922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-2fa1aeca-faf9-4555-ab2d-9c06503af2fb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 05:31:54.586922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:54.586922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:54.586922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:54.586922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:54.586922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:54.586922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:54.586922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:54.586922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 05:31:54.586922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:54.586922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:54.586922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:54.586922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:54.586922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:54.586922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:54.586922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:54.586922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:54.588135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:54.588135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:54.588135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:54.588135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:54.588135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:54.588135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:54.588135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:54.588135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:54.588135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:54.588135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:54.588135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:54.588135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:54.588135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:54.588135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:54.588135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:54.588135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:54.594233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:54.594233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:54.594233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:54.594233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:54.594233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:54.594233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:54.594233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:54.594233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:54.594233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:54.594233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:54.594233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:54.594233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:54.594233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 21 05:31:54.594233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:54.594233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 512/1014] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:54.597418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd0d41-1765-40a6-9c2e-1a170b120e2a None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:54.618867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db425045-9f7c-4085-82b7-beb1b63d7b0e None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:54.619193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db425045-9f7c-4085-82b7-beb1b63d7b0e None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:54.619471 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db425045-9f7c-4085-82b7-beb1b63d7b0e None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:54.621215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 513/1015] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:54.627064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e141e8-f9e2-4bd0-ae84-016889081adb None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:54.627614 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e141e8-f9e2-4bd0-ae84-016889081adb None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:54.627898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e141e8-f9e2-4bd0-ae84-016889081adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:54.652183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-94e141e8-f9e2-4bd0-ae84-016889081adb None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:54.653547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-94e141e8-f9e2-4bd0-ae84-016889081adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:54.669370 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:54.728499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd0d41-1765-40a6-9c2e-1a170b120e2a None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:54.732761 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 514/1016] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:54 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:54.755037 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e141e8-f9e2-4bd0-ae84-016889081adb None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:54.763489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8cd26-95ac-46c1-b3c8-65b173db45c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:54.764164 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8cd26-95ac-46c1-b3c8-65b173db45c4 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:54.764446 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8cd26-95ac-46c1-b3c8-65b173db45c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:54.764725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8cd26-95ac-46c1-b3c8-65b173db45c4 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:54.765677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8cd26-95ac-46c1-b3c8-65b173db45c4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:54.765804 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8cd26-95ac-46c1-b3c8-65b173db45c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:54.767121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8cd26-95ac-46c1-b3c8-65b173db45c4 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:54.776781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 503/1017] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jul 21 05:31:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:54.778618 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8cd26-95ac-46c1-b3c8-65b173db45c4 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:54.864077 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e141e8-f9e2-4bd0-ae84-016889081adb None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:54.878456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8cd26-95ac-46c1-b3c8-65b173db45c4 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:54.888741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:54.895889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:54.902514 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429ddda-521d-47f6-9dc7-9eaf809041af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:54.903217 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429ddda-521d-47f6-9dc7-9eaf809041af None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:54.903513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429ddda-521d-47f6-9dc7-9eaf809041af None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:54.903782 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429ddda-521d-47f6-9dc7-9eaf809041af None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:54.904507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429ddda-521d-47f6-9dc7-9eaf809041af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:54.905889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429ddda-521d-47f6-9dc7-9eaf809041af None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:54.916773 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c689c-2465-4c40-8d20-03dcbc213eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:54.917621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c689c-2465-4c40-8d20-03dcbc213eeb None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:54.917990 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c689c-2465-4c40-8d20-03dcbc213eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:54.918348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c689c-2465-4c40-8d20-03dcbc213eeb None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:54.919157 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c689c-2465-4c40-8d20-03dcbc213eeb None admin] RBAC: Adding query filter params (type=compute) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:54.919525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c689c-2465-4c40-8d20-03dcbc213eeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:54.920945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c689c-2465-4c40-8d20-03dcbc213eeb None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:54.923044 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 515/1018] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jul 21 05:31:54 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:31:54.926317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 504/1019] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jul 21 05:31:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:54.932312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429ddda-521d-47f6-9dc7-9eaf809041af None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:55.055294 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429ddda-521d-47f6-9dc7-9eaf809041af None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:55.091383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c732dea-a882-4577-b269-b906ef7baec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:55.092194 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c732dea-a882-4577-b269-b906ef7baec4 None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:55.092573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c732dea-a882-4577-b269-b906ef7baec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:55.092966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c732dea-a882-4577-b269-b906ef7baec4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:55.093795 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c732dea-a882-4577-b269-b906ef7baec4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:55.095300 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c732dea-a882-4577-b269-b906ef7baec4 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:55.118387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 516/1020] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jul 21 05:31:54 2022] POST /identity/v3/endpoints => generated 334 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:31:56.546706 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee4f6f-a280-4130-9210-db5cb5a59016 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:56.547300 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee4f6f-a280-4130-9210-db5cb5a59016 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:56.547387 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee4f6f-a280-4130-9210-db5cb5a59016 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:56.548777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 505/1021] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:56.554186 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd08b9-cb17-4d42-91a0-7f3d46d2d382 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:56.554622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd08b9-cb17-4d42-91a0-7f3d46d2d382 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:56.554703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd08b9-cb17-4d42-91a0-7f3d46d2d382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:56.572414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-6abd08b9-cb17-4d42-91a0-7f3d46d2d382 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:56.573586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-6abd08b9-cb17-4d42-91a0-7f3d46d2d382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:56.600727 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:56.654166 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 517/1022] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:56.680611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee4f6f-a280-4130-9210-db5cb5a59016 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:56.785823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee4f6f-a280-4130-9210-db5cb5a59016 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:56.819672 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff2b68-3505-40dd-890c-c6d3472b07ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:56.820388 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff2b68-3505-40dd-890c-c6d3472b07ba None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:56.820665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff2b68-3505-40dd-890c-c6d3472b07ba None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:56.820932 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff2b68-3505-40dd-890c-c6d3472b07ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:56.822113 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff2b68-3505-40dd-890c-c6d3472b07ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:56.823577 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff2b68-3505-40dd-890c-c6d3472b07ba None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:56.825356 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 506/1023] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:56.831097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd08b9-cb17-4d42-91a0-7f3d46d2d382 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:56.934116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd08b9-cb17-4d42-91a0-7f3d46d2d382 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:56.968201 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad59e9-16eb-4ab1-824d-b59dc8437cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:56.968931 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad59e9-16eb-4ab1-824d-b59dc8437cac None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:56.969213 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad59e9-16eb-4ab1-824d-b59dc8437cac None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:56.969483 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad59e9-16eb-4ab1-824d-b59dc8437cac None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:56.970204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad59e9-16eb-4ab1-824d-b59dc8437cac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:56.971682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad59e9-16eb-4ab1-824d-b59dc8437cac None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:56.999040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 518/1024] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:31:56 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:31:56.999276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34765224-95a1-4c9c-96fc-d127e64f2b19 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:56.999276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34765224-95a1-4c9c-96fc-d127e64f2b19 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:56.999276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34765224-95a1-4c9c-96fc-d127e64f2b19 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:56.999276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 507/1025] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:57.004285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62511a0-0df7-4800-bd54-792311f23d7b None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:57.004631 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62511a0-0df7-4800-bd54-792311f23d7b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:57.004918 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62511a0-0df7-4800-bd54-792311f23d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:57.023075 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-d62511a0-0df7-4800-bd54-792311f23d7b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:57.024299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d62511a0-0df7-4800-bd54-792311f23d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:57.036514 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:57.102685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 519/1026] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:57.125178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34765224-95a1-4c9c-96fc-d127e64f2b19 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:57.179565 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:57.221267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb1fd2-ca78-445c-b42b-112fd50cdb7e None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:57.221581 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb1fd2-ca78-445c-b42b-112fd50cdb7e None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:57.221863 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb1fd2-ca78-445c-b42b-112fd50cdb7e None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:57.223513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 520/1027] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:57.230398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401c543-ce80-49f1-b2f3-90d1349697ff None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:57.230818 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401c543-ce80-49f1-b2f3-90d1349697ff None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:57.231246 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401c543-ce80-49f1-b2f3-90d1349697ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:57.236914 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34765224-95a1-4c9c-96fc-d127e64f2b19 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:57.259432 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-9401c543-ce80-49f1-b2f3-90d1349697ff None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:57.260774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9401c543-ce80-49f1-b2f3-90d1349697ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:57.271725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfcd08e-4649-4e1e-9723-3424257564ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:57.272525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfcd08e-4649-4e1e-9723-3424257564ae None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:57.272806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfcd08e-4649-4e1e-9723-3424257564ae None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:57.273077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfcd08e-4649-4e1e-9723-3424257564ae None admin] PATH_INFO: `/v3/services/image` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:57.273729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfcd08e-4649-4e1e-9723-3424257564ae None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:57.275156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfcd08e-4649-4e1e-9723-3424257564ae None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:57.287688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-1cfcd08e-4649-4e1e-9723-3424257564ae None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 05:31:57.287688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:57.287688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:31:57.287688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:31:57.287688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:31:57.287688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:31:57.287688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:31:57.287688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:57.287688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 05:31:57.287688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:57.287688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:31:57.287688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:57.287688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:57.287688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:57.287688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:57.287688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:57.287688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:57.288909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:57.288909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:57.288909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:57.288909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:57.288909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:57.288909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:57.288909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:31:57.288909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:31:57.288909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:31:57.288909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:31:57.288909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:57.288909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:57.288909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:57.288909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:57.288909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:57.288909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:57.290450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:57.290450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:57.290450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:57.290450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:57.290450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:57.290450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:57.290450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:57.290450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:57.290450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:57.290450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:57.290450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:31:57.290450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:31:57.290450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 05:31:57.290450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:57.290450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 508/1028] 10.222.0.44 () {56 vars in 1105 bytes} [Thu Jul 21 05:31:57 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:57.294986 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfcd08e-4649-4e1e-9723-3424257564ae None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:57.333408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 521/1029] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:31:57.357107 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401c543-ce80-49f1-b2f3-90d1349697ff None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:57.402034 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfcd08e-4649-4e1e-9723-3424257564ae None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:57.416487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:57.443590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cbd29a-cf71-4529-8291-f449bec05bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:57.444278 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbd29a-cf71-4529-8291-f449bec05bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:57.444563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbd29a-cf71-4529-8291-f449bec05bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:57.444846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbd29a-cf71-4529-8291-f449bec05bc7 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:57.445528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:57.445643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cbd29a-cf71-4529-8291-f449bec05bc7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:57.445856 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cbd29a-cf71-4529-8291-f449bec05bc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:57.447398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cbd29a-cf71-4529-8291-f449bec05bc7 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:57.453022 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 509/1030] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:57.458581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cbd29a-cf71-4529-8291-f449bec05bc7 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:57.474051 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401c543-ce80-49f1-b2f3-90d1349697ff None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:57.510100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680551a-90de-4f1e-ae57-4042558a5013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:57.510965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680551a-90de-4f1e-ae57-4042558a5013 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:57.511280 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680551a-90de-4f1e-ae57-4042558a5013 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:57.511553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680551a-90de-4f1e-ae57-4042558a5013 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:57.523002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680551a-90de-4f1e-ae57-4042558a5013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:57.531539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-7680551a-90de-4f1e-ae57-4042558a5013 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 05:31:57.531539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:57.531539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:57.531539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:57.531539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:57.531539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:57.531539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:57.531539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:57.531539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:57.531539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:57.531539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:57.531539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:57.531539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:57.531539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:57.531539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:57.531539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:57.532509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:57.532509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:57.532509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:57.532509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:57.532509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:57.532509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:57.532509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:57.532509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:57.532509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:57.532509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:57.532509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:57.532509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:57.532509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:57.532509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:57.532509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:57.532509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:57.533370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:57.533370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:57.533370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:57.533370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:57.533370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:57.533370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:57.533370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:57.533370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:57.533370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:57.533370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:57.533370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 05:31:57.533370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:57.533370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 522/1031] 10.222.0.44 () {56 vars in 1120 bytes} [Thu Jul 21 05:31:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:57.539225 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680551a-90de-4f1e-ae57-4042558a5013 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:57.565612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cbd29a-cf71-4529-8291-f449bec05bc7 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:57.603003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed56a9-dfe2-430f-b3c8-9fe9bdf7010d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:57.603673 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed56a9-dfe2-430f-b3c8-9fe9bdf7010d None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:57.603974 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed56a9-dfe2-430f-b3c8-9fe9bdf7010d None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:57.604252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed56a9-dfe2-430f-b3c8-9fe9bdf7010d None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:57.604959 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed56a9-dfe2-430f-b3c8-9fe9bdf7010d None admin] RBAC: Adding query filter params (type=image) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:57.605275 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed56a9-dfe2-430f-b3c8-9fe9bdf7010d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:57.606646 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed56a9-dfe2-430f-b3c8-9fe9bdf7010d None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:57.617121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 510/1032] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:57.618078 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed56a9-dfe2-430f-b3c8-9fe9bdf7010d None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:57.643573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680551a-90de-4f1e-ae57-4042558a5013 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:57.667881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:57.687827 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67a595-39b1-40fb-98e2-63b473eec1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:57.688510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67a595-39b1-40fb-98e2-63b473eec1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:57.688793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67a595-39b1-40fb-98e2-63b473eec1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:57.689066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67a595-39b1-40fb-98e2-63b473eec1c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:57.689810 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67a595-39b1-40fb-98e2-63b473eec1c4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:57.690120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67a595-39b1-40fb-98e2-63b473eec1c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:57.691541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67a595-39b1-40fb-98e2-63b473eec1c4 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:57.694273 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:57.707699 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 523/1033] 10.222.0.44 () {56 vars in 1115 bytes} [Thu Jul 21 05:31:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:57.708531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67a595-39b1-40fb-98e2-63b473eec1c4 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:57.735190 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed56a9-dfe2-430f-b3c8-9fe9bdf7010d None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:57.771005 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925415d7-abb8-404f-a50a-f213cbb77ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:57.771738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925415d7-abb8-404f-a50a-f213cbb77ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:57.772067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925415d7-abb8-404f-a50a-f213cbb77ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:57.772345 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925415d7-abb8-404f-a50a-f213cbb77ce8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:57.773139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925415d7-abb8-404f-a50a-f213cbb77ce8 None admin] RBAC: Adding query filter params (interface=public, service_id=88b62106ad83400bbe047db2c3a3bda8, region_id=RegionOne) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:57.773447 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925415d7-abb8-404f-a50a-f213cbb77ce8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:57.774826 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925415d7-abb8-404f-a50a-f213cbb77ce8 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:57.784695 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 511/1034] 10.222.0.44 () {56 vars in 1251 bytes} [Thu Jul 21 05:31:57 2022] GET /identity/v3/endpoints?service_id=88b62106ad83400bbe047db2c3a3bda8&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:57.785666 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925415d7-abb8-404f-a50a-f213cbb77ce8 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:57.821755 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67a595-39b1-40fb-98e2-63b473eec1c4 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:57.860749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f05fb-67e3-41a3-be5f-029dc34cb7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:57.861671 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f05fb-67e3-41a3-be5f-029dc34cb7fd None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:57.862180 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f05fb-67e3-41a3-be5f-029dc34cb7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:57.862180 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f05fb-67e3-41a3-be5f-029dc34cb7fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:57.872534 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f05fb-67e3-41a3-be5f-029dc34cb7fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:57.873918 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f05fb-67e3-41a3-be5f-029dc34cb7fd None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:57.885386 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925415d7-abb8-404f-a50a-f213cbb77ce8 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:57.888421 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:57.891496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-a42f05fb-67e3-41a3-be5f-029dc34cb7fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:57.891496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:57.891496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:57.891496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:57.891496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:57.891496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:57.891496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:57.891496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:57.891496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:57.891496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:57.891496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:57.891496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:57.891496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:57.891496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:57.891496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:57.892443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:57.892443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:57.892443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:57.892443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:57.892443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:57.892443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:57.892443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:57.892443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:57.892443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:57.892443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:57.892443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:57.892443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:57.892443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:57.892443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:57.892443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:57.892443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:57.892443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:57.898555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:57.898555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:57.898555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:57.898555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:57.898555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 524/1035] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:57.899225 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f05fb-67e3-41a3-be5f-029dc34cb7fd None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:57.920980 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc712530-f9d7-4f52-999e-652a616be37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:57.921652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc712530-f9d7-4f52-999e-652a616be37c None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:57.922002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc712530-f9d7-4f52-999e-652a616be37c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:57.922203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc712530-f9d7-4f52-999e-652a616be37c None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:57.922879 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc712530-f9d7-4f52-999e-652a616be37c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:57.924282 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc712530-f9d7-4f52-999e-652a616be37c None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:57.939549 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 512/1036] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jul 21 05:31:57 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:31:57.952144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:58.007422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f05fb-67e3-41a3-be5f-029dc34cb7fd None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:58.045316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d57c1d-30f6-4329-9207-849147426930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:58.046000 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d57c1d-30f6-4329-9207-849147426930 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:58.046283 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d57c1d-30f6-4329-9207-849147426930 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:58.046553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d57c1d-30f6-4329-9207-849147426930 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:58.047322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d57c1d-30f6-4329-9207-849147426930 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:58.047628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d57c1d-30f6-4329-9207-849147426930 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:58.049027 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d57c1d-30f6-4329-9207-849147426930 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:58.061774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 525/1037] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:58.067661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc712530-f9d7-4f52-999e-652a616be37c None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:58.173626 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc712530-f9d7-4f52-999e-652a616be37c None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:58.211798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525d791-681e-4865-8164-be1f79703f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:58.212606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525d791-681e-4865-8164-be1f79703f55 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:58.212887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525d791-681e-4865-8164-be1f79703f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:58.213170 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525d791-681e-4865-8164-be1f79703f55 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:58.220756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0525d791-681e-4865-8164-be1f79703f55 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:58.222200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0525d791-681e-4865-8164-be1f79703f55 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:58.236800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-0525d791-681e-4865-8164-be1f79703f55 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:31:58.236800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:58.236800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:58.236800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:58.236800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:58.236800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:58.236800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:58.236800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:58.236800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:58.236800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:58.236800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:58.236800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:58.236800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:31:58.236800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:31:58.236800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:31:58.236800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:58.237780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:58.238765 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:58.238765 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:58.238765 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:31:58.238765 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:31:58.238765 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:31:58.238765 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:31:58.238765 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:31:58.238765 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:31:58.238765 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:31:58.238765 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:31:58.238765 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 513/1038] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jul 21 05:31:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:58.243846 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d57c1d-30f6-4329-9207-849147426930 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:58.345810 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d57c1d-30f6-4329-9207-849147426930 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:58.357713 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:58.384092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697031b-ac25-4f95-a7a9-97fa6089e276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:58.385031 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697031b-ac25-4f95-a7a9-97fa6089e276 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:58.385454 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697031b-ac25-4f95-a7a9-97fa6089e276 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:58.385867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7697031b-ac25-4f95-a7a9-97fa6089e276 None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:58.386837 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697031b-ac25-4f95-a7a9-97fa6089e276 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:58.387309 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697031b-ac25-4f95-a7a9-97fa6089e276 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:58.388886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7697031b-ac25-4f95-a7a9-97fa6089e276 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:58.406996 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 526/1039] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jul 21 05:31:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:58.412949 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525d791-681e-4865-8164-be1f79703f55 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:58.443174 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:58.516311 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525d791-681e-4865-8164-be1f79703f55 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:58.549958 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ab8f1-506c-4857-85f7-046409660367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:58.550611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ab8f1-506c-4857-85f7-046409660367 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:58.550887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ab8f1-506c-4857-85f7-046409660367 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:58.551186 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ab8f1-506c-4857-85f7-046409660367 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:58.617702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ab8f1-506c-4857-85f7-046409660367 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:58.619202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ab8f1-506c-4857-85f7-046409660367 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:58.649344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:58.733834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 514/1040] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:58 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 05:31:58.739521 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697031b-ac25-4f95-a7a9-97fa6089e276 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:58.837835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7697031b-ac25-4f95-a7a9-97fa6089e276 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:58.870675 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caee551-a8b8-464a-8e7f-88074f53c560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:58.871507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caee551-a8b8-464a-8e7f-88074f53c560 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:58.871783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caee551-a8b8-464a-8e7f-88074f53c560 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:58.872053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caee551-a8b8-464a-8e7f-88074f53c560 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:58.882244 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caee551-a8b8-464a-8e7f-88074f53c560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:58.883727 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caee551-a8b8-464a-8e7f-88074f53c560 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:58.900969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-9caee551-a8b8-464a-8e7f-88074f53c560 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:58.900969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:58.900969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:58.900969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:58.900969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:58.900969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:58.900969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:58.900969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:58.900969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:58.900969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:58.900969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:58.900969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:58.900969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:31:58.900969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:31:58.900969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:31:58.901887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:31:58.901887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:58.901887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:58.901887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:58.901887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:58.901887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:58.901887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:58.901887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:58.901887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:58.901887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:58.901887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:58.901887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:58.901887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:58.901887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:58.901887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:58.901887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:58.901887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:58.902806 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:31:58.902806 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:31:58.902806 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:31:58.902806 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:58.903642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 527/1041] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:58.907919 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ab8f1-506c-4857-85f7-046409660367 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:59.009440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ab8f1-506c-4857-85f7-046409660367 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:59.040207 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:59.049291 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3596fe-6e70-4a23-a044-49bd9a25a969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:59.049965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3596fe-6e70-4a23-a044-49bd9a25a969 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:59.050242 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3596fe-6e70-4a23-a044-49bd9a25a969 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:59.050509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3596fe-6e70-4a23-a044-49bd9a25a969 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:59.051270 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3596fe-6e70-4a23-a044-49bd9a25a969 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:59.051574 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3596fe-6e70-4a23-a044-49bd9a25a969 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:59.052972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3596fe-6e70-4a23-a044-49bd9a25a969 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:59.064894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 515/1042] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:59.070570 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caee551-a8b8-464a-8e7f-88074f53c560 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:59.092856 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:59.120023 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62864107-e8c1-4dd7-bb24-4d60a40960b2 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:59.120361 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62864107-e8c1-4dd7-bb24-4d60a40960b2 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:59.120689 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62864107-e8c1-4dd7-bb24-4d60a40960b2 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:59.122204 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 516/1043] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:31:59.127204 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd30b32-27eb-43e2-8af7-62dabfc2c482 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:59.127608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd30b32-27eb-43e2-8af7-62dabfc2c482 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:59.127889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd30b32-27eb-43e2-8af7-62dabfc2c482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:59.145868 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-0dd30b32-27eb-43e2-8af7-62dabfc2c482 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:59.147130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-0dd30b32-27eb-43e2-8af7-62dabfc2c482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:31:59.172136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caee551-a8b8-464a-8e7f-88074f53c560 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:59.206811 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075699a-422c-43f1-8369-c18982dd2b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:59.207674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075699a-422c-43f1-8369-c18982dd2b71 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:59.207960 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075699a-422c-43f1-8369-c18982dd2b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:59.208552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075699a-422c-43f1-8369-c18982dd2b71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:59.214902 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075699a-422c-43f1-8369-c18982dd2b71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:59.216388 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075699a-422c-43f1-8369-c18982dd2b71 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:59.223259 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 517/1044] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:31:59.229702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-a075699a-422c-43f1-8369-c18982dd2b71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:59.229702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:59.229702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:59.229702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:59.229702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:59.229702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:59.229702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:59.229702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:59.229702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:59.229702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:59.229702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:59.229702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:59.229702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:31:59.229702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:31:59.229702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:31:59.230599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:31:59.230599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:59.230599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:59.230599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:59.230599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:59.230599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:59.230599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:59.230599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:59.230599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:59.230599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:59.230599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:59.230599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:59.230599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:59.230599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:59.230599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:59.230599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:59.230599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:59.235830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:31:59.235830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:31:59.235830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:31:59.235830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:31:59.235830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:31:59.235830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:31:59.235830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:31:59.235830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:59.235830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 528/1045] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:31:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:31:59.238079 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd30b32-27eb-43e2-8af7-62dabfc2c482 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:59.246567 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075699a-422c-43f1-8369-c18982dd2b71 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:59.330106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:59.339933 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd30b32-27eb-43e2-8af7-62dabfc2c482 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:59.350363 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075699a-422c-43f1-8369-c18982dd2b71 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:59.359521 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:59.378180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229f601-c62f-4a6c-b925-f6c434fa2527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:59.378883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229f601-c62f-4a6c-b925-f6c434fa2527 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:59.379162 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229f601-c62f-4a6c-b925-f6c434fa2527 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:59.379631 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229f601-c62f-4a6c-b925-f6c434fa2527 None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:59.380241 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229f601-c62f-4a6c-b925-f6c434fa2527 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:59.380523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229f601-c62f-4a6c-b925-f6c434fa2527 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:59.382145 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229f601-c62f-4a6c-b925-f6c434fa2527 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:59.385174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd2405-c0f5-4823-84a2-948ae9fd4be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:59.385985 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd2405-c0f5-4823-84a2-948ae9fd4be3 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:59.386268 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd2405-c0f5-4823-84a2-948ae9fd4be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:59.386542 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd2405-c0f5-4823-84a2-948ae9fd4be3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:59.394438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 518/1046] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:31:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:59.397505 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd2405-c0f5-4823-84a2-948ae9fd4be3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:31:59.401042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229f601-c62f-4a6c-b925-f6c434fa2527 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:59.405357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-3bbd2405-c0f5-4823-84a2-948ae9fd4be3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:31:59.405357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:31:59.405357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:31:59.405357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:31:59.405357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:31:59.405357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:31:59.405357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:31:59.405357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:31:59.405357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:31:59.405357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:31:59.405357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:31:59.405357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:31:59.405357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:31:59.405357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:31:59.405357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:31:59.405357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:31:59.406479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:31:59.406479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:31:59.406479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:31:59.406479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:31:59.406479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:31:59.406479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:31:59.406479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:31:59.406479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:31:59.406479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:31:59.406479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:31:59.406479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:31:59.406479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:31:59.406479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:31:59.406479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:31:59.406479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:31:59.406479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:31:59.407743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:31:59.407743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:31:59.407743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:31:59.407743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:31:59.407743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:31:59.407743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:31:59.407743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:31:59.407743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:31:59.407743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:31:59.407743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:31:59.407743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:31:59.407743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:31:59.407743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 529/1047] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:31:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:31:59.412149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd2405-c0f5-4823-84a2-948ae9fd4be3 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:59.514648 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229f601-c62f-4a6c-b925-f6c434fa2527 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:59.515918 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd2405-c0f5-4823-84a2-948ae9fd4be3 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:59.550525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922f0e8-4439-4acf-9600-be7de18e42a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:59.550933 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35e47d-16cd-4050-96e4-2bd8c90b1f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:59.551222 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922f0e8-4439-4acf-9600-be7de18e42a4 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:59.551507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922f0e8-4439-4acf-9600-be7de18e42a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:59.551622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35e47d-16cd-4050-96e4-2bd8c90b1f5a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:59.551806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922f0e8-4439-4acf-9600-be7de18e42a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:59.551904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35e47d-16cd-4050-96e4-2bd8c90b1f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:59.552155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35e47d-16cd-4050-96e4-2bd8c90b1f5a None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:59.552622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922f0e8-4439-4acf-9600-be7de18e42a4 None admin] RBAC: Adding query filter params (role.id=545b77cdab164cd29c9b639b341c4018, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=928d014648cd41d7b58e7e01845eb748) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:59.552948 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35e47d-16cd-4050-96e4-2bd8c90b1f5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:31:59.553066 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922f0e8-4439-4acf-9600-be7de18e42a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:59.553249 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35e47d-16cd-4050-96e4-2bd8c90b1f5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:59.554467 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922f0e8-4439-4acf-9600-be7de18e42a4 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:59.554637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35e47d-16cd-4050-96e4-2bd8c90b1f5a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:59.571060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 519/1048] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:31:59 2022] GET /identity/v3/role_assignments?user.id=928d014648cd41d7b58e7e01845eb748&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=545b77cdab164cd29c9b639b341c4018 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:59.571315 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 530/1049] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:31:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:31:59.571415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922f0e8-4439-4acf-9600-be7de18e42a4 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:59.617435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:59.688458 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922f0e8-4439-4acf-9600-be7de18e42a4 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:59.736016 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79fdef5-07a1-4081-a8cc-b1d17559af9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:59.736915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fdef5-07a1-4081-a8cc-b1d17559af9e None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:59.737197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fdef5-07a1-4081-a8cc-b1d17559af9e None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:59.737466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79fdef5-07a1-4081-a8cc-b1d17559af9e None admin] PATH_INFO: `/v3/users/c5e33262e9a2489189ce32b76598bfd8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:59.751733 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fdef5-07a1-4081-a8cc-b1d17559af9e None admin] RBAC: Authorizing `identity:get_user(user_id=c5e33262e9a2489189ce32b76598bfd8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:59.753188 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79fdef5-07a1-4081-a8cc-b1d17559af9e None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:59.755074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 520/1050] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:31:59 2022] GET /identity/v3/users/c5e33262e9a2489189ce32b76598bfd8 => generated 285 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:31:59.761529 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35e47d-16cd-4050-96e4-2bd8c90b1f5a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:31:59.765751 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:59.869203 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35e47d-16cd-4050-96e4-2bd8c90b1f5a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:31:59.902517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b99e1-44c4-4946-a781-52198461a092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:31:59.903230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b99e1-44c4-4946-a781-52198461a092 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:59.903512 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b99e1-44c4-4946-a781-52198461a092 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:59.903783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b99e1-44c4-4946-a781-52198461a092 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:59.952988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e0e30-3815-4fa9-947d-88a320e5ae16 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:59.953310 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e0e30-3815-4fa9-947d-88a320e5ae16 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:59.953553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e0e30-3815-4fa9-947d-88a320e5ae16 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:59.955105 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 521/1051] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:31:59.960067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2f191-4416-45da-9dcc-c4650507bc5e None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:31:59.960367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2f191-4416-45da-9dcc-c4650507bc5e None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:31:59.960672 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2f191-4416-45da-9dcc-c4650507bc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:31:59.971546 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b99e1-44c4-4946-a781-52198461a092 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:31:59.973027 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b99e1-44c4-4946-a781-52198461a092 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:31:59.975346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:31:59.978303 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-6fd2f191-4416-45da-9dcc-c4650507bc5e None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:31:59.979494 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6fd2f191-4416-45da-9dcc-c4650507bc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:00.050796 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 522/1052] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:31:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:32:00.075734 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2f191-4416-45da-9dcc-c4650507bc5e None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:00.079415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 531/1053] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:31:59 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 05:32:00.085328 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b99e1-44c4-4946-a781-52198461a092 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:00.112600 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:00.188698 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b99e1-44c4-4946-a781-52198461a092 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:00.190535 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:00.191173 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2f191-4416-45da-9dcc-c4650507bc5e None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:00.226767 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b61348-04c8-412a-82ca-8c37a319844f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:00.227676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b61348-04c8-412a-82ca-8c37a319844f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:00.227958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b61348-04c8-412a-82ca-8c37a319844f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:00.228235 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b61348-04c8-412a-82ca-8c37a319844f None admin] PATH_INFO: `/v3/projects/09c0bc907a1b4877a66c57b8deb45d80` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:00.228486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf23d3d-d575-4867-a842-f926f419c3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:00.229294 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf23d3d-d575-4867-a842-f926f419c3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:00.229392 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b61348-04c8-412a-82ca-8c37a319844f None admin] RBAC: Authorizing `identity:get_project(project_id=09c0bc907a1b4877a66c57b8deb45d80)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:00.229575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf23d3d-d575-4867-a842-f926f419c3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:00.229852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf23d3d-d575-4867-a842-f926f419c3c6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:00.230517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf23d3d-d575-4867-a842-f926f419c3c6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:00.230856 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b61348-04c8-412a-82ca-8c37a319844f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:00.231898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf23d3d-d575-4867-a842-f926f419c3c6 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:00.237530 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 532/1054] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:32:00 2022] GET /identity/v3/projects/09c0bc907a1b4877a66c57b8deb45d80 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:32:00.238545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b61348-04c8-412a-82ca-8c37a319844f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:00.244116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-baf23d3d-d575-4867-a842-f926f419c3c6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 05:32:00.244116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:00.244116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:32:00.244116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:32:00.244116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:32:00.244116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:32:00.244116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:32:00.244116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:32:00.244116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 05:32:00.244116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:32:00.244116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:32:00.244116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:32:00.244116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:00.244116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:00.244116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:00.244116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:00.244116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:00.245918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:00.245918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:00.245918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:00.245918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:00.245918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:00.245918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:00.245918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:32:00.245918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:32:00.245918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:32:00.245918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:32:00.245918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:00.245918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:00.245918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:00.245918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:00.245918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:00.245918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:00.246771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:00.246771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:00.246771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:00.246771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:00.246771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:00.246771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:00.246771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:00.246771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:00.246771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:00.246771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:00.246771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:32:00.246771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:32:00.246771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 21 05:32:00.246771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:32:00.246771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 523/1055] 10.222.0.44 () {56 vars in 1105 bytes} [Thu Jul 21 05:32:00 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:32:00.253834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf23d3d-d575-4867-a842-f926f419c3c6 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:00.343423 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b61348-04c8-412a-82ca-8c37a319844f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:00.355853 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:00.363886 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf23d3d-d575-4867-a842-f926f419c3c6 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:00.377233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee1725-588b-4da0-95f8-159bb5651771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:00.377886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee1725-588b-4da0-95f8-159bb5651771 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:00.378377 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee1725-588b-4da0-95f8-159bb5651771 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:00.378487 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee1725-588b-4da0-95f8-159bb5651771 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:00.379272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee1725-588b-4da0-95f8-159bb5651771 None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=09c0bc907a1b4877a66c57b8deb45d80, user.id=c5e33262e9a2489189ce32b76598bfd8) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:00.379573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee1725-588b-4da0-95f8-159bb5651771 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:00.381166 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee1725-588b-4da0-95f8-159bb5651771 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:00.387874 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 533/1056] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:32:00 2022] GET /identity/v3/role_assignments?user.id=c5e33262e9a2489189ce32b76598bfd8&scope.project.id=09c0bc907a1b4877a66c57b8deb45d80&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:32:00.399393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e9cba-9630-43a9-ad2e-a08787be1f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:00.400102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e9cba-9630-43a9-ad2e-a08787be1f62 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:00.400390 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e9cba-9630-43a9-ad2e-a08787be1f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:00.400666 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e9cba-9630-43a9-ad2e-a08787be1f62 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:00.401402 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e9cba-9630-43a9-ad2e-a08787be1f62 None admin] RBAC: Adding query filter params (name=image) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:00.401715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e9cba-9630-43a9-ad2e-a08787be1f62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:00.403182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e9cba-9630-43a9-ad2e-a08787be1f62 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:00.416655 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 524/1057] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:32:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:32:00.418472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee1725-588b-4da0-95f8-159bb5651771 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:00.458025 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:00.531572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee1725-588b-4da0-95f8-159bb5651771 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:00.570016 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7500576-81a0-45f1-ba26-e37f1a08ff92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:00.570690 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7500576-81a0-45f1-ba26-e37f1a08ff92 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:00.570981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7500576-81a0-45f1-ba26-e37f1a08ff92 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:00.571360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7500576-81a0-45f1-ba26-e37f1a08ff92 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:00.572061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7500576-81a0-45f1-ba26-e37f1a08ff92 None admin] RBAC: Adding query filter params (type=image) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:00.572364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7500576-81a0-45f1-ba26-e37f1a08ff92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:00.573733 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7500576-81a0-45f1-ba26-e37f1a08ff92 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:00.578841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 534/1058] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:32:00 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:00.584577 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e9cba-9630-43a9-ad2e-a08787be1f62 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:00.699933 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e9cba-9630-43a9-ad2e-a08787be1f62 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:00.735496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25518490-ea6f-4abe-9da3-77e1d514c511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:00.736067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25518490-ea6f-4abe-9da3-77e1d514c511 None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:00.736302 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25518490-ea6f-4abe-9da3-77e1d514c511 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:00.736610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25518490-ea6f-4abe-9da3-77e1d514c511 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:00.737370 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25518490-ea6f-4abe-9da3-77e1d514c511 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:00.738742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25518490-ea6f-4abe-9da3-77e1d514c511 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:00.758563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 525/1059] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jul 21 05:32:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:32:01.575005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84131b8e-266e-4408-9e63-d936ed92c6e1 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:01.575654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84131b8e-266e-4408-9e63-d936ed92c6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:01.575654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84131b8e-266e-4408-9e63-d936ed92c6e1 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:01.577140 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 535/1060] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:01.582085 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2d66f-075d-4831-a600-babce404d56b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:01.582343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2d66f-075d-4831-a600-babce404d56b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:01.582656 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2d66f-075d-4831-a600-babce404d56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:01.599535 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-3ed2d66f-075d-4831-a600-babce404d56b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:01.600734 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-3ed2d66f-075d-4831-a600-babce404d56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:01.609892 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:01.673188 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 526/1061] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:01.695524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84131b8e-266e-4408-9e63-d936ed92c6e1 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:01.799997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:01.805345 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84131b8e-266e-4408-9e63-d936ed92c6e1 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:01.840275 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df145acd-0dfc-4098-aa1f-8dfd1ae45c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:01.841022 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df145acd-0dfc-4098-aa1f-8dfd1ae45c56 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:01.841306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df145acd-0dfc-4098-aa1f-8dfd1ae45c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:01.841583 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df145acd-0dfc-4098-aa1f-8dfd1ae45c56 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:01.852131 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df145acd-0dfc-4098-aa1f-8dfd1ae45c56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:32:01.860466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-df145acd-0dfc-4098-aa1f-8dfd1ae45c56 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 05:32:01.860466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:01.860466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:01.860466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:01.860466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:01.860466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:01.860466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:01.860466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:01.860466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:01.860466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:01.860466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:01.860466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:01.860466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:32:01.860466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:32:01.860466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:32:01.860466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:32:01.861409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:32:01.861409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:32:01.861409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:32:01.861409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:32:01.861409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:01.861409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:01.861409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:01.861409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:01.861409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:01.861409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:01.861409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:01.861409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:01.861409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:01.861409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:01.861409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:01.861409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:01.862269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:01.862269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:01.862269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:01.862269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:01.862269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:32:01.862269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:32:01.862269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:32:01.862269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:32:01.862269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:32:01.862269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:32:01.862269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 05:32:01.862269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:32:01.862269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 536/1062] 10.222.0.44 () {56 vars in 1120 bytes} [Thu Jul 21 05:32:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:32:01.869517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2d66f-075d-4831-a600-babce404d56b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:01.975600 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2d66f-075d-4831-a600-babce404d56b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:02.011305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f12730-6b76-4ef4-8c8a-b99a6e1be0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:02.011983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f12730-6b76-4ef4-8c8a-b99a6e1be0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:02.012254 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f12730-6b76-4ef4-8c8a-b99a6e1be0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:02.012531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f12730-6b76-4ef4-8c8a-b99a6e1be0a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:02.013289 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f12730-6b76-4ef4-8c8a-b99a6e1be0a1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:02.013604 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f12730-6b76-4ef4-8c8a-b99a6e1be0a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:02.014997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f12730-6b76-4ef4-8c8a-b99a6e1be0a1 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:02.023676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 527/1063] 10.222.0.44 () {56 vars in 1115 bytes} [Thu Jul 21 05:32:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:02.029407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df145acd-0dfc-4098-aa1f-8dfd1ae45c56 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:02.129509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df145acd-0dfc-4098-aa1f-8dfd1ae45c56 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:02.162973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95c3cd-9dc1-4ef2-8562-516eca3f3373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:02.163805 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95c3cd-9dc1-4ef2-8562-516eca3f3373 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:02.164082 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95c3cd-9dc1-4ef2-8562-516eca3f3373 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:02.164358 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95c3cd-9dc1-4ef2-8562-516eca3f3373 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:02.174945 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95c3cd-9dc1-4ef2-8562-516eca3f3373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:02.176439 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95c3cd-9dc1-4ef2-8562-516eca3f3373 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:02.193224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-6c95c3cd-9dc1-4ef2-8562-516eca3f3373 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:32:02.193224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:02.193224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:02.193224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:02.193224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:02.193224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:02.193224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:02.193224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:02.193224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:02.193224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:02.193224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:02.193224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:02.193224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:32:02.193224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:32:02.193224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:32:02.194209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:32:02.194209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:02.194209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:02.194209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:02.194209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:02.194209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:02.194209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:02.194209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:02.194209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:02.194209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:02.194209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:02.194209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:02.194209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:02.194209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:02.194209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:02.194209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:02.194209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:02.195184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:32:02.195184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:32:02.195184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:32:02.195184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:32:02.200888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 537/1064] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:32:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:32:02.201020 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f12730-6b76-4ef4-8c8a-b99a6e1be0a1 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:02.203778 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:02.308232 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f12730-6b76-4ef4-8c8a-b99a6e1be0a1 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:02.343551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e95f48-6bba-442b-a22b-4bda3085bece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:02.344213 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e95f48-6bba-442b-a22b-4bda3085bece None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:02.344495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e95f48-6bba-442b-a22b-4bda3085bece None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:02.344753 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e95f48-6bba-442b-a22b-4bda3085bece None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:02.345473 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e95f48-6bba-442b-a22b-4bda3085bece None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:02.345783 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e95f48-6bba-442b-a22b-4bda3085bece None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:02.347182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e95f48-6bba-442b-a22b-4bda3085bece None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:02.360448 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 528/1065] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:02.366583 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95c3cd-9dc1-4ef2-8562-516eca3f3373 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:02.368406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9e6d7-3f63-48b6-baa6-b7e9a9f59497 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:02.368700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9e6d7-3f63-48b6-baa6-b7e9a9f59497 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:02.368982 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9e6d7-3f63-48b6-baa6-b7e9a9f59497 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:02.370351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 529/1066] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:32:02.375367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572e230-2efd-4f4a-b1f0-0de7db0c27d0 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:02.375662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572e230-2efd-4f4a-b1f0-0de7db0c27d0 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:02.375942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572e230-2efd-4f4a-b1f0-0de7db0c27d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:02.380660 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:02.394100 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-d572e230-2efd-4f4a-b1f0-0de7db0c27d0 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:02.395338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d572e230-2efd-4f4a-b1f0-0de7db0c27d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:02.457125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:02.468832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 530/1067] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:02.481590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95c3cd-9dc1-4ef2-8562-516eca3f3373 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:02.491314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572e230-2efd-4f4a-b1f0-0de7db0c27d0 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:02.518225 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec4b5c-f810-4964-9b33-918a0e781700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:02.519260 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec4b5c-f810-4964-9b33-918a0e781700 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:02.519597 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec4b5c-f810-4964-9b33-918a0e781700 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:02.519870 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec4b5c-f810-4964-9b33-918a0e781700 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:02.529217 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec4b5c-f810-4964-9b33-918a0e781700 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:02.530698 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec4b5c-f810-4964-9b33-918a0e781700 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:02.545944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-a5ec4b5c-f810-4964-9b33-918a0e781700 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:32:02.545944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:02.545944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:02.545944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:02.545944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:02.545944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:02.545944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:02.545944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:02.545944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:02.545944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:02.545944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:02.545944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:02.545944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:32:02.545944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:32:02.545944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:32:02.545944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:02.546943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:02.547961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:02.547961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:02.547961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:32:02.547961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:32:02.547961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:32:02.547961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:32:02.547961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:32:02.547961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:32:02.547961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:32:02.547961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:32:02.547961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 538/1068] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jul 21 05:32:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:32:02.553216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec4b5c-f810-4964-9b33-918a0e781700 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:02.596993 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572e230-2efd-4f4a-b1f0-0de7db0c27d0 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:02.628718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:02.634861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d1931-0554-4ac8-9760-f7271a1e60d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:02.636213 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d1931-0554-4ac8-9760-f7271a1e60d5 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:02.636606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d1931-0554-4ac8-9760-f7271a1e60d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:02.636976 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d1931-0554-4ac8-9760-f7271a1e60d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:02.649134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d1931-0554-4ac8-9760-f7271a1e60d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:32:02.656205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-485d1931-0554-4ac8-9760-f7271a1e60d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:32:02.656205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:02.656205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:02.656205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:02.656205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:02.656205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:02.656205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:02.656205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:02.656205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:02.656205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:02.656205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:02.656205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:02.656205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:32:02.656205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:32:02.656205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:32:02.656205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:32:02.657351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:32:02.657351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:32:02.657351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:32:02.657351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:32:02.657351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:02.657351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:02.657351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:02.657351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:02.657351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:02.657351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:02.657351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:02.657351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:02.657351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:02.657351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:02.657351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:02.657351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:02.658497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:02.658497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:02.658497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:02.658497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:02.658497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:32:02.658497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:32:02.658497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:32:02.658497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:32:02.658497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:32:02.658497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:32:02.658497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:32:02.658497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:32:02.660602 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 531/1069] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:32:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:32:02.667110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d1931-0554-4ac8-9760-f7271a1e60d5 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:02.667591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec4b5c-f810-4964-9b33-918a0e781700 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:02.707353 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc717ea-a549-43f9-ae02-a2384ddb713a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:02.708131 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc717ea-a549-43f9-ae02-a2384ddb713a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:02.708484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc717ea-a549-43f9-ae02-a2384ddb713a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:02.708827 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc717ea-a549-43f9-ae02-a2384ddb713a None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:02.709776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc717ea-a549-43f9-ae02-a2384ddb713a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:02.710174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc717ea-a549-43f9-ae02-a2384ddb713a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:02.711792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc717ea-a549-43f9-ae02-a2384ddb713a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:02.723560 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:02.736192 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 539/1070] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jul 21 05:32:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:02.737200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc717ea-a549-43f9-ae02-a2384ddb713a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:02.774051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d1931-0554-4ac8-9760-f7271a1e60d5 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:02.812421 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eda12d-eaf6-47cd-8873-d94899ef01b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:02.813105 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eda12d-eaf6-47cd-8873-d94899ef01b6 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:02.813388 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eda12d-eaf6-47cd-8873-d94899ef01b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:02.813661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eda12d-eaf6-47cd-8873-d94899ef01b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:02.814415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eda12d-eaf6-47cd-8873-d94899ef01b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:02.814720 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eda12d-eaf6-47cd-8873-d94899ef01b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:02.816144 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eda12d-eaf6-47cd-8873-d94899ef01b6 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:02.827058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 532/1071] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:32:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:32:02.830768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51990c-cf3a-472a-ba04-0d710b9c77f0 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:02.831079 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51990c-cf3a-472a-ba04-0d710b9c77f0 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:02.831357 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51990c-cf3a-472a-ba04-0d710b9c77f0 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:02.832654 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 533/1072] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:32:02.833337 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51990c-cf3a-472a-ba04-0d710b9c77f0 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:02.844977 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc717ea-a549-43f9-ae02-a2384ddb713a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:02.870482 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:02.878772 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803874c-00b0-49cf-b800-b634e8fdb027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:02.879613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803874c-00b0-49cf-b800-b634e8fdb027 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:02.879898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803874c-00b0-49cf-b800-b634e8fdb027 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:02.880178 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803874c-00b0-49cf-b800-b634e8fdb027 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:02.890179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803874c-00b0-49cf-b800-b634e8fdb027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:02.891615 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803874c-00b0-49cf-b800-b634e8fdb027 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:02.908357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-c803874c-00b0-49cf-b800-b634e8fdb027 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:32:02.908357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:02.908357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:02.908357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:02.908357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:02.908357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:02.908357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:02.908357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:02.908357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:02.908357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:02.908357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:02.908357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:02.908357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:32:02.908357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:32:02.908357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:32:02.909311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:32:02.909311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:02.909311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:02.909311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:02.909311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:02.909311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:02.909311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:02.909311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:02.909311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:02.909311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:02.909311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:02.909311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:02.909311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:02.909311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:02.909311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:02.909311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:02.909311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:02.910225 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:32:02.910225 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:32:02.910225 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:32:02.910225 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:32:02.910225 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 540/1073] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:32:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:32:02.910671 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f27fa-77e8-4930-88a4-2adfcf9bbf18 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:02.910950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f27fa-77e8-4930-88a4-2adfcf9bbf18 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:02.911247 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f27fa-77e8-4930-88a4-2adfcf9bbf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:02.927554 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-d31f27fa-77e8-4930-88a4-2adfcf9bbf18 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:02.928760 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d31f27fa-77e8-4930-88a4-2adfcf9bbf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:02.934421 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51990c-cf3a-472a-ba04-0d710b9c77f0 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:02.948457 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:02.969470 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dd175-6064-4643-a594-79ae4db11a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:02.970352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd175-6064-4643-a594-79ae4db11a82 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:02.970640 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd175-6064-4643-a594-79ae4db11a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:02.970918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd175-6064-4643-a594-79ae4db11a82 None admin] PATH_INFO: `/v3/users/c5e33262e9a2489189ce32b76598bfd8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:02.972163 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dd175-6064-4643-a594-79ae4db11a82 None admin] RBAC: Authorizing `identity:get_user(user_id=c5e33262e9a2489189ce32b76598bfd8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:02.973607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dd175-6064-4643-a594-79ae4db11a82 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:02.975349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 534/1074] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:32:02 2022] GET /identity/v3/users/c5e33262e9a2489189ce32b76598bfd8 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:02.976108 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dd175-6064-4643-a594-79ae4db11a82 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:02.997617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 541/1075] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:02.998461 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f27fa-77e8-4930-88a4-2adfcf9bbf18 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:03.080693 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dd175-6064-4643-a594-79ae4db11a82 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:03.094450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f27fa-77e8-4930-88a4-2adfcf9bbf18 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:03.104668 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:03.116322 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd98d32-a93a-49d6-a7d4-16fdc8eebd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:03.117003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd98d32-a93a-49d6-a7d4-16fdc8eebd4b None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:03.117289 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd98d32-a93a-49d6-a7d4-16fdc8eebd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:03.117555 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd98d32-a93a-49d6-a7d4-16fdc8eebd4b None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:03.118294 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd98d32-a93a-49d6-a7d4-16fdc8eebd4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:03.118604 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd98d32-a93a-49d6-a7d4-16fdc8eebd4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:03.120024 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd98d32-a93a-49d6-a7d4-16fdc8eebd4b None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:03.128540 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516e10e-da82-439d-a2a7-7ef20f46fedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:03.129422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516e10e-da82-439d-a2a7-7ef20f46fedb None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:03.129707 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516e10e-da82-439d-a2a7-7ef20f46fedb None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:03.129986 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516e10e-da82-439d-a2a7-7ef20f46fedb None admin] PATH_INFO: `/v3/projects/09c0bc907a1b4877a66c57b8deb45d80` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:03.131242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516e10e-da82-439d-a2a7-7ef20f46fedb None admin] RBAC: Authorizing `identity:get_project(project_id=09c0bc907a1b4877a66c57b8deb45d80)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:03.131353 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 535/1076] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:03.132031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd98d32-a93a-49d6-a7d4-16fdc8eebd4b None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:03.132736 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516e10e-da82-439d-a2a7-7ef20f46fedb None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:03.134688 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 542/1077] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:32:02 2022] GET /identity/v3/projects/09c0bc907a1b4877a66c57b8deb45d80 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:03.137028 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516e10e-da82-439d-a2a7-7ef20f46fedb None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:03.177161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:03.231924 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd98d32-a93a-49d6-a7d4-16fdc8eebd4b None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:03.233625 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516e10e-da82-439d-a2a7-7ef20f46fedb None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:03.264375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca11afdc-1f9d-4805-9347-790eaabd3529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:03.265017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11afdc-1f9d-4805-9347-790eaabd3529 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:03.265298 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11afdc-1f9d-4805-9347-790eaabd3529 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:03.265573 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11afdc-1f9d-4805-9347-790eaabd3529 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:03.269530 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69783aed-aa39-433c-950a-406a0c0a350c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:03.270327 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69783aed-aa39-433c-950a-406a0c0a350c None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:03.270609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69783aed-aa39-433c-950a-406a0c0a350c None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:03.270883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69783aed-aa39-433c-950a-406a0c0a350c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:03.277978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69783aed-aa39-433c-950a-406a0c0a350c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:03.279478 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69783aed-aa39-433c-950a-406a0c0a350c None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:03.293433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-69783aed-aa39-433c-950a-406a0c0a350c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:32:03.293433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:03.293433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:03.293433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:03.293433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:03.293433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:03.293433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:03.293433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:03.293433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:03.293433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:03.293433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:03.293433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:03.293433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:32:03.293433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:32:03.293433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:32:03.294504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:32:03.294504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:03.294504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:03.294504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:03.294504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:03.294504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:03.294504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:03.294504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:03.294504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:03.294504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:03.294504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:03.294504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:03.294504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:03.294504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:03.294504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:03.294504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:03.294504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:03.300695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:32:03.300695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:32:03.300695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:32:03.300695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:32:03.300695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:32:03.300695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:32:03.300695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:32:03.300695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:32:03.300695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 543/1078] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:32:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:32:03.300695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69783aed-aa39-433c-950a-406a0c0a350c None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:03.332916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca11afdc-1f9d-4805-9347-790eaabd3529 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:03.334464 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca11afdc-1f9d-4805-9347-790eaabd3529 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:03.334844 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:03.374625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:03.393078 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69783aed-aa39-433c-950a-406a0c0a350c None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:03.426461 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c5376-d40f-4116-9580-0687069147bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:03.427444 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c5376-d40f-4116-9580-0687069147bb None admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:03.427730 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c5376-d40f-4116-9580-0687069147bb None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:03.428014 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c5376-d40f-4116-9580-0687069147bb None admin] PATH_INFO: `/v3/projects/09c0bc907a1b4877a66c57b8deb45d80/users/c5e33262e9a2489189ce32b76598bfd8/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:03.430071 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c5376-d40f-4116-9580-0687069147bb None admin] RBAC: Authorizing `identity:create_grant(project_id=09c0bc907a1b4877a66c57b8deb45d80, user_id=c5e33262e9a2489189ce32b76598bfd8, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:03.431499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 536/1079] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:32:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 05:32:03.431634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c5376-d40f-4116-9580-0687069147bb None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:03.432339 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca11afdc-1f9d-4805-9347-790eaabd3529 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:03.449219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 544/1080] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:32:03 2022] PUT /identity/v3/projects/09c0bc907a1b4877a66c57b8deb45d80/users/c5e33262e9a2489189ce32b76598bfd8/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:32:03.450391 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c5376-d40f-4116-9580-0687069147bb None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:03.560006 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:03.561311 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca11afdc-1f9d-4805-9347-790eaabd3529 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:03.565143 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c5376-d40f-4116-9580-0687069147bb None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:03.604267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd54e0-cf3e-444e-a1de-b8aedb86c851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:03.604758 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758ee7f-db8d-464f-8611-e8d31492518a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:03.605443 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758ee7f-db8d-464f-8611-e8d31492518a None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:03.605725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758ee7f-db8d-464f-8611-e8d31492518a None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:03.605938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd54e0-cf3e-444e-a1de-b8aedb86c851 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:03.606035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758ee7f-db8d-464f-8611-e8d31492518a None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:03.606414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd54e0-cf3e-444e-a1de-b8aedb86c851 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:03.606776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd54e0-cf3e-444e-a1de-b8aedb86c851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:03.606864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758ee7f-db8d-464f-8611-e8d31492518a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:03.608265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758ee7f-db8d-464f-8611-e8d31492518a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:03.608586 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758ee7f-db8d-464f-8611-e8d31492518a None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:03.621033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd54e0-cf3e-444e-a1de-b8aedb86c851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:03.622100 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 537/1081] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:32:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:03.622494 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd54e0-cf3e-444e-a1de-b8aedb86c851 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:03.628624 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758ee7f-db8d-464f-8611-e8d31492518a None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:03.631765 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:03.640297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-0dfd54e0-cf3e-444e-a1de-b8aedb86c851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:32:03.640297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:03.640297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:03.640297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:03.640297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:03.640297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:03.640297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:03.640297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:03.640297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:03.640297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:03.640297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:03.640297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:03.640297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:32:03.640297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:32:03.640297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:32:03.641683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:32:03.641683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:03.641683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:03.641683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:03.641683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:03.641683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:03.641683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:03.641683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:03.641683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:03.641683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:03.641683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:03.641683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:03.641683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:03.641683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:03.641683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:03.641683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:03.641683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:03.642764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:32:03.642764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:32:03.642764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:32:03.642764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:32:03.642764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 545/1082] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:32:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:32:03.648001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd54e0-cf3e-444e-a1de-b8aedb86c851 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:03.753314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758ee7f-db8d-464f-8611-e8d31492518a None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:03.754791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd54e0-cf3e-444e-a1de-b8aedb86c851 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:03.787130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268badc-869c-403d-823e-04594c9367e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:03.788393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268badc-869c-403d-823e-04594c9367e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:03.788665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268badc-869c-403d-823e-04594c9367e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:03.788936 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268badc-869c-403d-823e-04594c9367e1 None admin] PATH_INFO: `/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/928d014648cd41d7b58e7e01845eb748/roles/545b77cdab164cd29c9b639b341c4018` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:03.791199 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268badc-869c-403d-823e-04594c9367e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b6f27b8d9b44d191210a24a9e04041, user_id=928d014648cd41d7b58e7e01845eb748, role_id=545b77cdab164cd29c9b639b341c4018)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:03.792570 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423d3db-1496-4722-8602-a0dc254ffed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:03.792713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268badc-869c-403d-823e-04594c9367e1 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:03.793514 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423d3db-1496-4722-8602-a0dc254ffed3 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:03.793950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423d3db-1496-4722-8602-a0dc254ffed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:03.794365 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423d3db-1496-4722-8602-a0dc254ffed3 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:03.795817 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423d3db-1496-4722-8602-a0dc254ffed3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:03.796274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423d3db-1496-4722-8602-a0dc254ffed3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:03.798059 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423d3db-1496-4722-8602-a0dc254ffed3 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:03.808106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 538/1083] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:32:03 2022] PUT /identity/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/928d014648cd41d7b58e7e01845eb748/roles/545b77cdab164cd29c9b639b341c4018 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:32:03.819049 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 546/1084] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:32:05.496615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c1aa2-e752-4143-ac74-6de0965c5b2c None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:05.497179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c1aa2-e752-4143-ac74-6de0965c5b2c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:05.497381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c1aa2-e752-4143-ac74-6de0965c5b2c None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:05.499346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 539/1085] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:05.504800 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a5e9c-dbed-45b0-bfdf-b6f71ce2e016 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:05.505256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a5e9c-dbed-45b0-bfdf-b6f71ce2e016 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:05.505349 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a5e9c-dbed-45b0-bfdf-b6f71ce2e016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:05.522089 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-f67a5e9c-dbed-45b0-bfdf-b6f71ce2e016 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:05.523389 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f67a5e9c-dbed-45b0-bfdf-b6f71ce2e016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:05.526277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:05.592863 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 547/1086] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:05.616939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c1aa2-e752-4143-ac74-6de0965c5b2c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:05.690067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:05.730030 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c1aa2-e752-4143-ac74-6de0965c5b2c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:05.764733 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512f2f3-7830-4af8-a652-43f598af5bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:05.765456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512f2f3-7830-4af8-a652-43f598af5bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:05.765739 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512f2f3-7830-4af8-a652-43f598af5bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:05.766012 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512f2f3-7830-4af8-a652-43f598af5bf9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:05.777086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8512f2f3-7830-4af8-a652-43f598af5bf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:32:05.784621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-8512f2f3-7830-4af8-a652-43f598af5bf9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:32:05.784621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:05.784621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:05.784621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:05.784621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:05.784621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:05.784621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:05.784621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:05.784621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:05.784621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:05.784621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:05.784621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:05.784621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:32:05.784621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:32:05.784621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:32:05.784621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:32:05.785772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:32:05.785772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:32:05.785772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:32:05.785772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:32:05.785772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:05.785772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:05.785772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:05.785772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:05.785772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:05.785772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:05.785772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:05.785772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:05.785772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:05.785772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:05.785772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:05.785772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:05.786651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:05.786651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:05.786651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:05.786651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:05.786651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:32:05.786651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:32:05.786651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:32:05.786651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:32:05.786651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:32:05.786651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:32:05.786651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:32:05.786651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:32:05.786651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 540/1087] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:32:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:32:05.792743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a5e9c-dbed-45b0-bfdf-b6f71ce2e016 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:05.816062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea56f9a-5e06-4b03-9dad-4764c1e9ef8d None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:05.816343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea56f9a-5e06-4b03-9dad-4764c1e9ef8d None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:05.816614 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea56f9a-5e06-4b03-9dad-4764c1e9ef8d None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:05.818134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 541/1088] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:32:05.823124 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1be1eb-37bb-46da-803e-1380e3ab19e3 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:05.823417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1be1eb-37bb-46da-803e-1380e3ab19e3 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:05.823690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1be1eb-37bb-46da-803e-1380e3ab19e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:05.841346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-3d1be1eb-37bb-46da-803e-1380e3ab19e3 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:05.842531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-3d1be1eb-37bb-46da-803e-1380e3ab19e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:05.894807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a5e9c-dbed-45b0-bfdf-b6f71ce2e016 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:05.911289 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 542/1089] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:05.927752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585c573-9e47-4634-9fd9-71fad4aa7169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:05.928423 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585c573-9e47-4634-9fd9-71fad4aa7169 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:05.928705 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585c573-9e47-4634-9fd9-71fad4aa7169 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:05.928979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585c573-9e47-4634-9fd9-71fad4aa7169 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:05.929714 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b585c573-9e47-4634-9fd9-71fad4aa7169 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:05.930021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b585c573-9e47-4634-9fd9-71fad4aa7169 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:05.931425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b585c573-9e47-4634-9fd9-71fad4aa7169 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:05.933660 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1be1eb-37bb-46da-803e-1380e3ab19e3 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:05.941666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 548/1090] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:32:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:05.947761 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585c573-9e47-4634-9fd9-71fad4aa7169 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:05.950511 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:05.987496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:06.033295 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1be1eb-37bb-46da-803e-1380e3ab19e3 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:06.048348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585c573-9e47-4634-9fd9-71fad4aa7169 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:06.066935 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea53a9b-b083-4379-89a2-b9fba5be816b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:06.067792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea53a9b-b083-4379-89a2-b9fba5be816b None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:06.068076 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea53a9b-b083-4379-89a2-b9fba5be816b None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:06.068383 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea53a9b-b083-4379-89a2-b9fba5be816b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:06.078424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea53a9b-b083-4379-89a2-b9fba5be816b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:32:06.081193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be615e24-fafb-4c4a-a92c-1513ff2ea030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:06.082073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be615e24-fafb-4c4a-a92c-1513ff2ea030 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:06.082356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be615e24-fafb-4c4a-a92c-1513ff2ea030 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:06.082629 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be615e24-fafb-4c4a-a92c-1513ff2ea030 None admin] PATH_INFO: `/v3/users/c5e33262e9a2489189ce32b76598bfd8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:06.083936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be615e24-fafb-4c4a-a92c-1513ff2ea030 None admin] RBAC: Authorizing `identity:get_user(user_id=c5e33262e9a2489189ce32b76598bfd8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:06.085391 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be615e24-fafb-4c4a-a92c-1513ff2ea030 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:06.086454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-fea53a9b-b083-4379-89a2-b9fba5be816b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 05:32:06.086454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:06.086454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:06.086454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:06.086454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:06.086454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:06.086454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:06.086454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:06.086454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:06.086454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:06.086454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:06.086454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:06.086454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:32:06.086454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:32:06.086454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:32:06.086454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:32:06.087636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:32:06.087636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:32:06.087636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:32:06.087636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:32:06.087636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:06.087636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:06.087636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:06.087636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:06.087636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:06.087636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:06.087636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:06.087636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:06.087636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:06.087636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:06.087636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:06.087636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:06.088952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:06.088952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:06.088952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:06.088952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:06.088952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:32:06.088952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:32:06.088952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:32:06.088952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:32:06.088952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:32:06.088952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:32:06.088952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 21 05:32:06.088952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:32:06.089751 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 549/1091] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:32:05 2022] GET /identity/v3/users/c5e33262e9a2489189ce32b76598bfd8 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:32:06.089843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 543/1092] 10.222.0.44 () {56 vars in 1120 bytes} [Thu Jul 21 05:32:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:32:06.092861 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be615e24-fafb-4c4a-a92c-1513ff2ea030 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:06.093468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea53a9b-b083-4379-89a2-b9fba5be816b None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:06.158804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:06.188861 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be615e24-fafb-4c4a-a92c-1513ff2ea030 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:06.189612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea53a9b-b083-4379-89a2-b9fba5be816b None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:06.202593 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:06.221811 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9f021-b970-41dc-ac7b-7886b4db20e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:06.222458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9f021-b970-41dc-ac7b-7886b4db20e7 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:06.222734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9f021-b970-41dc-ac7b-7886b4db20e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:06.223005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe9f021-b970-41dc-ac7b-7886b4db20e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:06.223476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd3465-feda-4024-9d8f-55205af9f01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:06.224210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd3465-feda-4024-9d8f-55205af9f01f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:06.224587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd3465-feda-4024-9d8f-55205af9f01f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:06.224940 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd3465-feda-4024-9d8f-55205af9f01f None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:06.225755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd3465-feda-4024-9d8f-55205af9f01f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:06.226125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd3465-feda-4024-9d8f-55205af9f01f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:06.227547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd3465-feda-4024-9d8f-55205af9f01f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:06.235486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 544/1093] 10.222.0.44 () {56 vars in 1115 bytes} [Thu Jul 21 05:32:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:32:06.241203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd3465-feda-4024-9d8f-55205af9f01f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:06.289641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9f021-b970-41dc-ac7b-7886b4db20e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:06.291122 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe9f021-b970-41dc-ac7b-7886b4db20e7 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:06.338994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd3465-feda-4024-9d8f-55205af9f01f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:06.356356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:06.373046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137896f-9e8c-43dc-92f0-7a49b07a7328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:06.373845 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137896f-9e8c-43dc-92f0-7a49b07a7328 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:06.374131 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137896f-9e8c-43dc-92f0-7a49b07a7328 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:06.374409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137896f-9e8c-43dc-92f0-7a49b07a7328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:06.385296 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137896f-9e8c-43dc-92f0-7a49b07a7328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:06.386753 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137896f-9e8c-43dc-92f0-7a49b07a7328 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:06.388344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 550/1094] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:32:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 05:32:06.394197 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9f021-b970-41dc-ac7b-7886b4db20e7 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:06.408963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-d137896f-9e8c-43dc-92f0-7a49b07a7328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:32:06.408963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:06.408963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:06.408963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:06.408963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:06.408963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:06.408963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:06.408963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:06.408963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:06.408963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:06.408963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:06.408963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:06.408963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:32:06.408963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:32:06.408963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:32:06.410041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:32:06.410041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:06.410041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:06.410041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:06.410041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:06.410041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:06.410041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:06.410041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:06.410041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:06.410041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:06.410041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:06.410041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:06.410041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:06.410041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:06.410041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:06.410041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:06.410041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:06.411058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:32:06.411058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:32:06.411058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:32:06.411058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:32:06.411058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 545/1095] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:32:06.416632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137896f-9e8c-43dc-92f0-7a49b07a7328 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:06.444895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:06.495879 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe9f021-b970-41dc-ac7b-7886b4db20e7 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:06.515559 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137896f-9e8c-43dc-92f0-7a49b07a7328 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:06.530159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f14580-8304-4e1f-98e2-a8317aeeb741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:06.531018 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f14580-8304-4e1f-98e2-a8317aeeb741 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:06.531324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f14580-8304-4e1f-98e2-a8317aeeb741 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:06.531599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f14580-8304-4e1f-98e2-a8317aeeb741 None admin] PATH_INFO: `/v3/projects/09c0bc907a1b4877a66c57b8deb45d80` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:06.532766 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f14580-8304-4e1f-98e2-a8317aeeb741 None admin] RBAC: Authorizing `identity:get_project(project_id=09c0bc907a1b4877a66c57b8deb45d80)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:06.534213 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f14580-8304-4e1f-98e2-a8317aeeb741 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:06.535957 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 551/1096] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:32:06 2022] GET /identity/v3/projects/09c0bc907a1b4877a66c57b8deb45d80 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:32:06.542482 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f14580-8304-4e1f-98e2-a8317aeeb741 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:06.550594 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4253f4-93e7-4f16-9d9b-b412fa080b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:06.551301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4253f4-93e7-4f16-9d9b-b412fa080b9d None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:06.551582 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4253f4-93e7-4f16-9d9b-b412fa080b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:06.551851 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4253f4-93e7-4f16-9d9b-b412fa080b9d None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:06.552577 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4253f4-93e7-4f16-9d9b-b412fa080b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:06.552880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4253f4-93e7-4f16-9d9b-b412fa080b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:06.554243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4253f4-93e7-4f16-9d9b-b412fa080b9d None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:06.566197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 546/1097] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:06.571907 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4253f4-93e7-4f16-9d9b-b412fa080b9d None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:06.580147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:06.638887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f14580-8304-4e1f-98e2-a8317aeeb741 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:06.662959 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:06.678152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ddfde3-2cc1-4ae7-9553-5248da6aca84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:06.678820 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddfde3-2cc1-4ae7-9553-5248da6aca84 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:06.679142 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddfde3-2cc1-4ae7-9553-5248da6aca84 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:06.679420 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ddfde3-2cc1-4ae7-9553-5248da6aca84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:06.679661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4253f4-93e7-4f16-9d9b-b412fa080b9d None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:06.680245 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ddfde3-2cc1-4ae7-9553-5248da6aca84 None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=09c0bc907a1b4877a66c57b8deb45d80, user.id=c5e33262e9a2489189ce32b76598bfd8) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:06.680558 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ddfde3-2cc1-4ae7-9553-5248da6aca84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:06.682003 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ddfde3-2cc1-4ae7-9553-5248da6aca84 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:06.688118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 552/1098] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:32:06 2022] GET /identity/v3/role_assignments?user.id=c5e33262e9a2489189ce32b76598bfd8&scope.project.id=09c0bc907a1b4877a66c57b8deb45d80&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:32:06.716686 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbe78f-24e9-4539-83e4-851155f6449e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:06.717502 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbe78f-24e9-4539-83e4-851155f6449e None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:06.717779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbe78f-24e9-4539-83e4-851155f6449e None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:06.718047 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbe78f-24e9-4539-83e4-851155f6449e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:06.726781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbe78f-24e9-4539-83e4-851155f6449e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:06.728387 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbe78f-24e9-4539-83e4-851155f6449e None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:06.745435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-0fcbe78f-24e9-4539-83e4-851155f6449e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:32:06.745435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:06.745435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:06.745435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:06.745435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:06.745435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:06.745435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:06.745435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:06.745435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:06.745435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:06.745435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:06.745435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:06.745435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:32:06.745435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:32:06.745435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:32:06.745435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:06.746641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:06.747683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:06.747683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:06.747683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:32:06.747683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:32:06.747683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:32:06.747683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:32:06.747683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:32:06.747683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:32:06.747683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 21 05:32:06.747683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:32:06.747683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 547/1099] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jul 21 05:32:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:32:06.753553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ddfde3-2cc1-4ae7-9553-5248da6aca84 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:06.885220 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ddfde3-2cc1-4ae7-9553-5248da6aca84 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:06.886731 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:06.920770 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27002c50-07d0-465d-95c7-aa9020fab79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:06.921457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27002c50-07d0-465d-95c7-aa9020fab79c None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:06.921739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27002c50-07d0-465d-95c7-aa9020fab79c None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:06.922014 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27002c50-07d0-465d-95c7-aa9020fab79c None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:06.922851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27002c50-07d0-465d-95c7-aa9020fab79c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:06.923198 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27002c50-07d0-465d-95c7-aa9020fab79c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:06.924641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27002c50-07d0-465d-95c7-aa9020fab79c None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:06.941680 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 553/1100] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jul 21 05:32:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:06.947664 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbe78f-24e9-4539-83e4-851155f6449e None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:07.049445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbe78f-24e9-4539-83e4-851155f6449e None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:07.084305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca4e8f-9603-4193-a33f-c3d81d8b9c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:07.084993 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca4e8f-9603-4193-a33f-c3d81d8b9c88 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:07.085261 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca4e8f-9603-4193-a33f-c3d81d8b9c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:07.085539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca4e8f-9603-4193-a33f-c3d81d8b9c88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:07.101447 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:07.154971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca4e8f-9603-4193-a33f-c3d81d8b9c88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:07.156394 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca4e8f-9603-4193-a33f-c3d81d8b9c88 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:07.253231 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 548/1101] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:32:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 05:32:07.259295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27002c50-07d0-465d-95c7-aa9020fab79c None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:07.360317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27002c50-07d0-465d-95c7-aa9020fab79c None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:07.395159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7a2e5-f0bc-46e6-82d1-dd4848b8c2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:07.395950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7a2e5-f0bc-46e6-82d1-dd4848b8c2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:07.396252 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7a2e5-f0bc-46e6-82d1-dd4848b8c2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:07.396531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7a2e5-f0bc-46e6-82d1-dd4848b8c2a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:07.406741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7a2e5-f0bc-46e6-82d1-dd4848b8c2a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:07.408184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7a2e5-f0bc-46e6-82d1-dd4848b8c2a3 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:07.426557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-0da7a2e5-f0bc-46e6-82d1-dd4848b8c2a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:32:07.426557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:07.426557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:07.426557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:07.426557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:07.426557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:07.426557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:07.426557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:07.426557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:07.426557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:07.426557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:07.426557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:07.426557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:32:07.426557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:32:07.426557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:32:07.427596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:32:07.427596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:07.427596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:07.427596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:07.427596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:07.427596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:07.427596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:07.427596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:07.427596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:07.427596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:07.427596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:07.427596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:07.427596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:07.427596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:07.427596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:07.427596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:07.427596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:07.428497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:32:07.428497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:32:07.428497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:32:07.428497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:32:07.428497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 554/1102] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:32:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:32:07.433734 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca4e8f-9603-4193-a33f-c3d81d8b9c88 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:07.479314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:07.536120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca4e8f-9603-4193-a33f-c3d81d8b9c88 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:07.570698 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d6451-f2ba-48f0-8e81-67b268da0daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:07.571393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d6451-f2ba-48f0-8e81-67b268da0daf None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:07.571673 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d6451-f2ba-48f0-8e81-67b268da0daf None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:07.571943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d6451-f2ba-48f0-8e81-67b268da0daf None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:07.572666 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d6451-f2ba-48f0-8e81-67b268da0daf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:07.572969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d6451-f2ba-48f0-8e81-67b268da0daf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:07.574331 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d6451-f2ba-48f0-8e81-67b268da0daf None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:07.585775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 549/1103] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:07.591460 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7a2e5-f0bc-46e6-82d1-dd4848b8c2a3 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:07.610157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:07.696452 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7a2e5-f0bc-46e6-82d1-dd4848b8c2a3 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:07.731229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0275a7a-0439-453b-944a-a7f7f9cff38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:07.732034 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0275a7a-0439-453b-944a-a7f7f9cff38a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:07.732319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0275a7a-0439-453b-944a-a7f7f9cff38a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:07.732590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0275a7a-0439-453b-944a-a7f7f9cff38a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:07.739429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0275a7a-0439-453b-944a-a7f7f9cff38a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:07.740941 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0275a7a-0439-453b-944a-a7f7f9cff38a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:07.754579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-f0275a7a-0439-453b-944a-a7f7f9cff38a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:32:07.754579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:07.754579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:07.754579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:07.754579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:07.754579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:07.754579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:07.754579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:07.754579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:07.754579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:07.754579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:07.754579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:07.754579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:32:07.754579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:32:07.754579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:32:07.755669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:32:07.755669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:07.755669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:07.755669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:07.755669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:07.755669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:07.755669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:07.755669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:07.755669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:07.755669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:07.755669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:07.755669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:07.755669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:07.755669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:07.755669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:07.755669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:07.755669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:07.756605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:32:07.756605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:32:07.756605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:32:07.756605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:32:07.756605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:32:07.756605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:32:07.756605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:32:07.756605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:32:07.756605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 555/1104] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:32:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:32:07.762058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d6451-f2ba-48f0-8e81-67b268da0daf None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:07.867907 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d6451-f2ba-48f0-8e81-67b268da0daf None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:07.875439 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:07.904437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6dcd2d-058f-4399-8d7a-a8bff9792dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:07.905100 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dcd2d-058f-4399-8d7a-a8bff9792dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:07.905387 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dcd2d-058f-4399-8d7a-a8bff9792dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:07.905658 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6dcd2d-058f-4399-8d7a-a8bff9792dd8 None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:07.906450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6dcd2d-058f-4399-8d7a-a8bff9792dd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:07.906754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6dcd2d-058f-4399-8d7a-a8bff9792dd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:07.908226 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6dcd2d-058f-4399-8d7a-a8bff9792dd8 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:07.921480 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 550/1105] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:32:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:07.927786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0275a7a-0439-453b-944a-a7f7f9cff38a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:07.998839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:08.027752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0275a7a-0439-453b-944a-a7f7f9cff38a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:08.061565 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee2e2e-cd74-413b-bae2-9eca1922a64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:08.062274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee2e2e-cd74-413b-bae2-9eca1922a64c None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:08.062566 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee2e2e-cd74-413b-bae2-9eca1922a64c None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:08.062837 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee2e2e-cd74-413b-bae2-9eca1922a64c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:08.063688 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee2e2e-cd74-413b-bae2-9eca1922a64c None admin] RBAC: Adding query filter params (role.id=545b77cdab164cd29c9b639b341c4018, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=928d014648cd41d7b58e7e01845eb748) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:08.063993 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee2e2e-cd74-413b-bae2-9eca1922a64c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:08.065422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee2e2e-cd74-413b-bae2-9eca1922a64c None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:08.071377 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 556/1106] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:32:07 2022] GET /identity/v3/role_assignments?user.id=928d014648cd41d7b58e7e01845eb748&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=545b77cdab164cd29c9b639b341c4018 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:08.765129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747c0bf-fa23-4291-9ed2-1f974565778c None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:08.765669 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747c0bf-fa23-4291-9ed2-1f974565778c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:08.765669 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747c0bf-fa23-4291-9ed2-1f974565778c None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:08.767073 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 551/1107] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:08.772673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7f4af-bf28-4baf-8495-4b49c5009bc5 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:08.773124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7f4af-bf28-4baf-8495-4b49c5009bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:08.773221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7f4af-bf28-4baf-8495-4b49c5009bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:08.791402 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-bdf7f4af-bf28-4baf-8495-4b49c5009bc5 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:08.792610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-bdf7f4af-bf28-4baf-8495-4b49c5009bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:08.862601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 557/1108] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:08.887564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747c0bf-fa23-4291-9ed2-1f974565778c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:08.999727 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747c0bf-fa23-4291-9ed2-1f974565778c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:09.034761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18900eab-5468-4193-a670-0a22bc5159a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:09.035583 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18900eab-5468-4193-a670-0a22bc5159a2 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:09.035864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18900eab-5468-4193-a670-0a22bc5159a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:09.036139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18900eab-5468-4193-a670-0a22bc5159a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:09.037841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18900eab-5468-4193-a670-0a22bc5159a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:09.039315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18900eab-5468-4193-a670-0a22bc5159a2 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:09.040942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 552/1109] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:32:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:09.047204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7f4af-bf28-4baf-8495-4b49c5009bc5 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:09.113566 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:09.152133 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7f4af-bf28-4baf-8495-4b49c5009bc5 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:09.186938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f932a7ae-1473-43ab-98b2-d5dc1c6fc728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:09.187627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932a7ae-1473-43ab-98b2-d5dc1c6fc728 None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:09.187905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932a7ae-1473-43ab-98b2-d5dc1c6fc728 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:09.188178 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f932a7ae-1473-43ab-98b2-d5dc1c6fc728 None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:09.188906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f932a7ae-1473-43ab-98b2-d5dc1c6fc728 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:09.190340 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f932a7ae-1473-43ab-98b2-d5dc1c6fc728 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:09.213026 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 558/1110] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:32:09 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:32:11.240674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfafaea-ea92-45d2-8614-2888a868320b None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:11.241597 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfafaea-ea92-45d2-8614-2888a868320b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:11.241597 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfafaea-ea92-45d2-8614-2888a868320b None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:11.243895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 553/1111] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:11.250119 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2b9a5-5c62-486a-b8ee-e372aa9d88e0 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:11.250874 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2b9a5-5c62-486a-b8ee-e372aa9d88e0 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:11.251207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2b9a5-5c62-486a-b8ee-e372aa9d88e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:11.270260 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-07b2b9a5-5c62-486a-b8ee-e372aa9d88e0 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:11.271480 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-07b2b9a5-5c62-486a-b8ee-e372aa9d88e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:11.339513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 559/1112] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:11.369083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfafaea-ea92-45d2-8614-2888a868320b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:11.418588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:11.488269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfafaea-ea92-45d2-8614-2888a868320b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:11.523519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5100f-1cf2-4630-8a2b-179b63264a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:11.524303 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5100f-1cf2-4630-8a2b-179b63264a24 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:11.524586 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5100f-1cf2-4630-8a2b-179b63264a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:11.524861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5100f-1cf2-4630-8a2b-179b63264a24 None admin] PATH_INFO: `/v3/roles/a43ba93f358b4b1a8a445851a5a216a0` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:11.526387 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5100f-1cf2-4630-8a2b-179b63264a24 None admin] RBAC: Authorizing `identity:get_role(role_id=a43ba93f358b4b1a8a445851a5a216a0)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:11.527804 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5100f-1cf2-4630-8a2b-179b63264a24 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:11.529178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 554/1113] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:32:11 2022] GET /identity/v3/roles/a43ba93f358b4b1a8a445851a5a216a0 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:11.538963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2b9a5-5c62-486a-b8ee-e372aa9d88e0 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:11.601958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:11.649295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2b9a5-5c62-486a-b8ee-e372aa9d88e0 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:11.684414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e28ca-ed5e-40f5-9f4c-4d69a610338c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:11.685293 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e28ca-ed5e-40f5-9f4c-4d69a610338c None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:11.685573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e28ca-ed5e-40f5-9f4c-4d69a610338c None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:11.685838 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e28ca-ed5e-40f5-9f4c-4d69a610338c None admin] PATH_INFO: `/v3/users/e3fb0b51022641e680af07ea04dcf133` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:11.699448 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e28ca-ed5e-40f5-9f4c-4d69a610338c None admin] RBAC: Authorizing `identity:get_user(user_id=e3fb0b51022641e680af07ea04dcf133)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:11.700892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e28ca-ed5e-40f5-9f4c-4d69a610338c None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:11.702529 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 560/1114] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:32:11 2022] GET /identity/v3/users/e3fb0b51022641e680af07ea04dcf133 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:11.709129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5100f-1cf2-4630-8a2b-179b63264a24 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:11.825277 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5100f-1cf2-4630-8a2b-179b63264a24 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:11.827726 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:11.864812 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bf93c-51b9-4958-bf9a-bd4f857986df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:11.865510 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bf93c-51b9-4958-bf9a-bd4f857986df None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:11.865798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bf93c-51b9-4958-bf9a-bd4f857986df None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:11.866078 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bf93c-51b9-4958-bf9a-bd4f857986df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:11.942772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bf93c-51b9-4958-bf9a-bd4f857986df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:11.944345 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bf93c-51b9-4958-bf9a-bd4f857986df None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:12.044877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 555/1115] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:32:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 05:32:12.054516 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e28ca-ed5e-40f5-9f4c-4d69a610338c None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:12.169580 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e28ca-ed5e-40f5-9f4c-4d69a610338c None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:12.180184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:12.210431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187959be-8c25-48a1-b3e0-ca0a0d4fb49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:12.211352 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187959be-8c25-48a1-b3e0-ca0a0d4fb49d None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:12.211636 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187959be-8c25-48a1-b3e0-ca0a0d4fb49d None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:12.211913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187959be-8c25-48a1-b3e0-ca0a0d4fb49d None admin] PATH_INFO: `/v3/projects/09c0bc907a1b4877a66c57b8deb45d80` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:12.213118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187959be-8c25-48a1-b3e0-ca0a0d4fb49d None admin] RBAC: Authorizing `identity:get_project(project_id=09c0bc907a1b4877a66c57b8deb45d80)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:12.214587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187959be-8c25-48a1-b3e0-ca0a0d4fb49d None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:12.216366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 561/1116] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:32:12 2022] GET /identity/v3/projects/09c0bc907a1b4877a66c57b8deb45d80 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:12.222865 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bf93c-51b9-4958-bf9a-bd4f857986df None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:12.225093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:12.340597 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bf93c-51b9-4958-bf9a-bd4f857986df None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:12.378151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ae9a7-adff-4be1-978f-5f8012535035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:12.378805 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ae9a7-adff-4be1-978f-5f8012535035 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:12.379112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ae9a7-adff-4be1-978f-5f8012535035 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:12.379391 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ae9a7-adff-4be1-978f-5f8012535035 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:12.380225 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ae9a7-adff-4be1-978f-5f8012535035 None admin] RBAC: Adding query filter params (role.id=a43ba93f358b4b1a8a445851a5a216a0, scope.project.id=09c0bc907a1b4877a66c57b8deb45d80, user.id=e3fb0b51022641e680af07ea04dcf133) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:12.380512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ae9a7-adff-4be1-978f-5f8012535035 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:12.381982 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ae9a7-adff-4be1-978f-5f8012535035 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:12.387575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 556/1117] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:32:12 2022] GET /identity/v3/role_assignments?user.id=e3fb0b51022641e680af07ea04dcf133&scope.project.id=09c0bc907a1b4877a66c57b8deb45d80&role.id=a43ba93f358b4b1a8a445851a5a216a0 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:14.372805 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65ee3e-cc4d-4656-8bd6-abc0ab5dc4ba None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:14.373719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65ee3e-cc4d-4656-8bd6-abc0ab5dc4ba None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:14.373719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65ee3e-cc4d-4656-8bd6-abc0ab5dc4ba None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:14.375976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 562/1118] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:14.382164 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d51d2b-db6d-4994-9860-25c7f88bc431 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:14.382590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d51d2b-db6d-4994-9860-25c7f88bc431 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:14.382769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d51d2b-db6d-4994-9860-25c7f88bc431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:14.405499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-83d51d2b-db6d-4994-9860-25c7f88bc431 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:14.406685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-83d51d2b-db6d-4994-9860-25c7f88bc431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:14.468749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:14.477068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 557/1119] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:14.505022 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65ee3e-cc4d-4656-8bd6-abc0ab5dc4ba None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:14.615632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65ee3e-cc4d-4656-8bd6-abc0ab5dc4ba None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:14.649950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f920e1-b33f-4628-94d8-bb79c2e7896f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:14.650744 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f920e1-b33f-4628-94d8-bb79c2e7896f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:14.651044 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f920e1-b33f-4628-94d8-bb79c2e7896f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:14.651329 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f920e1-b33f-4628-94d8-bb79c2e7896f None admin] PATH_INFO: `/v3/roles/a43ba93f358b4b1a8a445851a5a216a0` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:14.652921 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f920e1-b33f-4628-94d8-bb79c2e7896f None admin] RBAC: Authorizing `identity:get_role(role_id=a43ba93f358b4b1a8a445851a5a216a0)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:14.654299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f920e1-b33f-4628-94d8-bb79c2e7896f None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:14.655723 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 563/1120] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:32:14 2022] GET /identity/v3/roles/a43ba93f358b4b1a8a445851a5a216a0 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:14.663833 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d51d2b-db6d-4994-9860-25c7f88bc431 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:14.767573 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d51d2b-db6d-4994-9860-25c7f88bc431 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:14.802605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aae6ce-2b09-4c2f-9eb3-9781c7c06b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:14.803556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aae6ce-2b09-4c2f-9eb3-9781c7c06b92 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:14.803840 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aae6ce-2b09-4c2f-9eb3-9781c7c06b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:14.804108 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aae6ce-2b09-4c2f-9eb3-9781c7c06b92 None admin] PATH_INFO: `/v3/users/e3fb0b51022641e680af07ea04dcf133` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:14.805321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aae6ce-2b09-4c2f-9eb3-9781c7c06b92 None admin] RBAC: Authorizing `identity:get_user(user_id=e3fb0b51022641e680af07ea04dcf133)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:14.806757 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aae6ce-2b09-4c2f-9eb3-9781c7c06b92 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:14.808403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 558/1121] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:32:14 2022] GET /identity/v3/users/e3fb0b51022641e680af07ea04dcf133 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:14.814448 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f920e1-b33f-4628-94d8-bb79c2e7896f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:14.863120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:14.918528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f920e1-b33f-4628-94d8-bb79c2e7896f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:14.953763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e07dfd-53b9-45cf-acac-cde947d5bf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:14.954675 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e07dfd-53b9-45cf-acac-cde947d5bf8e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:14.954959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e07dfd-53b9-45cf-acac-cde947d5bf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:14.955259 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e07dfd-53b9-45cf-acac-cde947d5bf8e None admin] PATH_INFO: `/v3/projects/09c0bc907a1b4877a66c57b8deb45d80` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:14.956545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e07dfd-53b9-45cf-acac-cde947d5bf8e None admin] RBAC: Authorizing `identity:get_project(project_id=09c0bc907a1b4877a66c57b8deb45d80)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:14.958007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e07dfd-53b9-45cf-acac-cde947d5bf8e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:14.959785 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 564/1122] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:32:14 2022] GET /identity/v3/projects/09c0bc907a1b4877a66c57b8deb45d80 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:14.966067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aae6ce-2b09-4c2f-9eb3-9781c7c06b92 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:15.034312 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:15.075188 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aae6ce-2b09-4c2f-9eb3-9781c7c06b92 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:15.110566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58581e76-9652-4487-a8da-76c79796e0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:15.111482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58581e76-9652-4487-a8da-76c79796e0f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:15.111757 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58581e76-9652-4487-a8da-76c79796e0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:15.112028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58581e76-9652-4487-a8da-76c79796e0f7 None admin] PATH_INFO: `/v3/projects/09c0bc907a1b4877a66c57b8deb45d80/users/e3fb0b51022641e680af07ea04dcf133/roles/a43ba93f358b4b1a8a445851a5a216a0` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:15.114577 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58581e76-9652-4487-a8da-76c79796e0f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=09c0bc907a1b4877a66c57b8deb45d80, user_id=e3fb0b51022641e680af07ea04dcf133, role_id=a43ba93f358b4b1a8a445851a5a216a0)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:15.116092 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58581e76-9652-4487-a8da-76c79796e0f7 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:15.135137 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 559/1123] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:32:14 2022] PUT /identity/v3/projects/09c0bc907a1b4877a66c57b8deb45d80/users/e3fb0b51022641e680af07ea04dcf133/roles/a43ba93f358b4b1a8a445851a5a216a0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:32:17.154917 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97452b20-c752-4ff1-aab5-70dc152bb55d None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:17.155479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97452b20-c752-4ff1-aab5-70dc152bb55d None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:17.155479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97452b20-c752-4ff1-aab5-70dc152bb55d None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:17.156796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 565/1124] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:17.163966 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea5c7e-c3aa-4bbc-9053-aab25e19857a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:17.164669 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea5c7e-c3aa-4bbc-9053-aab25e19857a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:17.164810 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea5c7e-c3aa-4bbc-9053-aab25e19857a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:17.192168 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-99ea5c7e-c3aa-4bbc-9053-aab25e19857a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:17.193784 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-99ea5c7e-c3aa-4bbc-9053-aab25e19857a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:17.270059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 560/1125] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:17.300481 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97452b20-c752-4ff1-aab5-70dc152bb55d None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:17.419510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97452b20-c752-4ff1-aab5-70dc152bb55d None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:17.433409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:17.454099 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45330d1-4933-4dbc-9fbe-cb7d2db5c0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:17.454956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45330d1-4933-4dbc-9fbe-cb7d2db5c0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:17.455264 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45330d1-4933-4dbc-9fbe-cb7d2db5c0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:17.455535 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45330d1-4933-4dbc-9fbe-cb7d2db5c0d1 None admin] PATH_INFO: `/v3/roles/a43ba93f358b4b1a8a445851a5a216a0` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:17.457105 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45330d1-4933-4dbc-9fbe-cb7d2db5c0d1 None admin] RBAC: Authorizing `identity:get_role(role_id=a43ba93f358b4b1a8a445851a5a216a0)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:17.458572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45330d1-4933-4dbc-9fbe-cb7d2db5c0d1 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:17.459979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 566/1126] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:32:17 2022] GET /identity/v3/roles/a43ba93f358b4b1a8a445851a5a216a0 => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:17.465983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea5c7e-c3aa-4bbc-9053-aab25e19857a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:17.525567 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:17.577279 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea5c7e-c3aa-4bbc-9053-aab25e19857a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:17.621818 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62143c56-5900-41b6-813b-2a8b5f41f8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:17.622700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62143c56-5900-41b6-813b-2a8b5f41f8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:17.622982 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62143c56-5900-41b6-813b-2a8b5f41f8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:17.623292 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62143c56-5900-41b6-813b-2a8b5f41f8b9 None admin] PATH_INFO: `/v3/users/e3fb0b51022641e680af07ea04dcf133` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:17.624631 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62143c56-5900-41b6-813b-2a8b5f41f8b9 None admin] RBAC: Authorizing `identity:get_user(user_id=e3fb0b51022641e680af07ea04dcf133)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:17.626086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62143c56-5900-41b6-813b-2a8b5f41f8b9 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:17.627804 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 561/1127] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:32:17 2022] GET /identity/v3/users/e3fb0b51022641e680af07ea04dcf133 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:17.634159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45330d1-4933-4dbc-9fbe-cb7d2db5c0d1 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:17.738013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45330d1-4933-4dbc-9fbe-cb7d2db5c0d1 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:17.780426 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba656bfb-0bd4-4369-b847-b63b74f18df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:17.781372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba656bfb-0bd4-4369-b847-b63b74f18df2 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:17.781755 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba656bfb-0bd4-4369-b847-b63b74f18df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:17.782105 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba656bfb-0bd4-4369-b847-b63b74f18df2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:17.831780 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:17.851700 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba656bfb-0bd4-4369-b847-b63b74f18df2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:17.853149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba656bfb-0bd4-4369-b847-b63b74f18df2 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:17.951035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 567/1128] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:32:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 05:32:17.960272 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62143c56-5900-41b6-813b-2a8b5f41f8b9 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:18.069027 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62143c56-5900-41b6-813b-2a8b5f41f8b9 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:18.093573 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:18.106655 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499eeb04-2055-41ca-8ac6-ddfd451452ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:18.107553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499eeb04-2055-41ca-8ac6-ddfd451452ee None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:18.107834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499eeb04-2055-41ca-8ac6-ddfd451452ee None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:18.108110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499eeb04-2055-41ca-8ac6-ddfd451452ee None admin] PATH_INFO: `/v3/projects/09c0bc907a1b4877a66c57b8deb45d80` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:18.109343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499eeb04-2055-41ca-8ac6-ddfd451452ee None admin] RBAC: Authorizing `identity:get_project(project_id=09c0bc907a1b4877a66c57b8deb45d80)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:18.110786 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499eeb04-2055-41ca-8ac6-ddfd451452ee None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:18.112507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 562/1129] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:32:17 2022] GET /identity/v3/projects/09c0bc907a1b4877a66c57b8deb45d80 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:18.119950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba656bfb-0bd4-4369-b847-b63b74f18df2 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:18.207575 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:18.226758 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba656bfb-0bd4-4369-b847-b63b74f18df2 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:18.261323 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92b6d1-8899-489f-a331-aaead30b7628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:18.261975 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92b6d1-8899-489f-a331-aaead30b7628 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:18.262257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92b6d1-8899-489f-a331-aaead30b7628 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:18.262534 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92b6d1-8899-489f-a331-aaead30b7628 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:18.263402 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92b6d1-8899-489f-a331-aaead30b7628 None admin] RBAC: Adding query filter params (role.id=a43ba93f358b4b1a8a445851a5a216a0, scope.project.id=09c0bc907a1b4877a66c57b8deb45d80, user.id=e3fb0b51022641e680af07ea04dcf133) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:18.263716 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92b6d1-8899-489f-a331-aaead30b7628 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:18.265147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92b6d1-8899-489f-a331-aaead30b7628 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:18.271346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 568/1130] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:32:18 2022] GET /identity/v3/role_assignments?user.id=e3fb0b51022641e680af07ea04dcf133&scope.project.id=09c0bc907a1b4877a66c57b8deb45d80&role.id=a43ba93f358b4b1a8a445851a5a216a0 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:20.313609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8947c1d-3640-426b-911d-39d356f0b8a3 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:20.314207 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8947c1d-3640-426b-911d-39d356f0b8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:20.314207 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8947c1d-3640-426b-911d-39d356f0b8a3 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:20.315591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 563/1131] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:20.321198 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6dab3-58f0-421f-859b-30f0abf9580e None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:20.321633 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6dab3-58f0-421f-859b-30f0abf9580e None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:20.321737 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6dab3-58f0-421f-859b-30f0abf9580e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:20.339945 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-0cf6dab3-58f0-421f-859b-30f0abf9580e None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:20.341122 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0cf6dab3-58f0-421f-859b-30f0abf9580e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:20.408913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 569/1132] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:20.439612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8947c1d-3640-426b-911d-39d356f0b8a3 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:20.546679 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8947c1d-3640-426b-911d-39d356f0b8a3 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:20.582820 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fffe1f-bef3-4c8b-aa39-76351ce304ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:20.583772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fffe1f-bef3-4c8b-aa39-76351ce304ee None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:20.584055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fffe1f-bef3-4c8b-aa39-76351ce304ee None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:20.584321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fffe1f-bef3-4c8b-aa39-76351ce304ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:20.585550 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fffe1f-bef3-4c8b-aa39-76351ce304ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:20.586984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fffe1f-bef3-4c8b-aa39-76351ce304ee None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:20.588614 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 564/1133] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:32:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:20.595241 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6dab3-58f0-421f-859b-30f0abf9580e None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:20.686843 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:20.715431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6dab3-58f0-421f-859b-30f0abf9580e None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:20.750121 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e9ebf-32fb-4d77-bc45-3d02bfe67342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:20.750776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9ebf-32fb-4d77-bc45-3d02bfe67342 None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:20.751062 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9ebf-32fb-4d77-bc45-3d02bfe67342 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:20.751348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9ebf-32fb-4d77-bc45-3d02bfe67342 None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:20.752054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e9ebf-32fb-4d77-bc45-3d02bfe67342 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:20.753619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e9ebf-32fb-4d77-bc45-3d02bfe67342 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:20.781304 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 570/1134] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jul 21 05:32:20 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:32:22.815580 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df4165-d3b7-4f90-b60c-dc223a836276 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:22.816160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df4165-d3b7-4f90-b60c-dc223a836276 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:22.816160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df4165-d3b7-4f90-b60c-dc223a836276 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:22.817526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 565/1135] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:22.824130 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233b1a7-0779-4b9b-9e68-6b8015350ab3 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:22.824837 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233b1a7-0779-4b9b-9e68-6b8015350ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:22.824991 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233b1a7-0779-4b9b-9e68-6b8015350ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:22.845950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-9233b1a7-0779-4b9b-9e68-6b8015350ab3 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:22.847195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9233b1a7-0779-4b9b-9e68-6b8015350ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:22.923967 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 571/1136] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:22.956189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df4165-d3b7-4f90-b60c-dc223a836276 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:23.022929 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:23.066794 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df4165-d3b7-4f90-b60c-dc223a836276 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:23.101719 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d640990-adb6-40a3-a83e-220c090a9f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:23.102498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d640990-adb6-40a3-a83e-220c090a9f49 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:23.102781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d640990-adb6-40a3-a83e-220c090a9f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:23.103045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d640990-adb6-40a3-a83e-220c090a9f49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:23.104370 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d640990-adb6-40a3-a83e-220c090a9f49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:23.105809 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d640990-adb6-40a3-a83e-220c090a9f49 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:23.107461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 566/1137] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:32:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:23.113666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233b1a7-0779-4b9b-9e68-6b8015350ab3 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:23.175987 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:23.232278 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233b1a7-0779-4b9b-9e68-6b8015350ab3 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:23.268100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d6ed2-953a-45ff-9457-8602684260d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:23.268769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d6ed2-953a-45ff-9457-8602684260d2 None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:23.269050 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d6ed2-953a-45ff-9457-8602684260d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:23.269321 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d6ed2-953a-45ff-9457-8602684260d2 None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:23.270029 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d6ed2-953a-45ff-9457-8602684260d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:23.271538 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d6ed2-953a-45ff-9457-8602684260d2 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:23.294946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 572/1138] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:32:23 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:32:25.335546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae492e8e-b799-4cfd-abe2-606e36766e2f None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:25.336439 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae492e8e-b799-4cfd-abe2-606e36766e2f None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:25.336439 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae492e8e-b799-4cfd-abe2-606e36766e2f None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:25.338634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 567/1139] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:25.344936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ceb50c-b713-41df-812e-2156581a4937 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:25.345703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ceb50c-b713-41df-812e-2156581a4937 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:25.345826 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ceb50c-b713-41df-812e-2156581a4937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:25.364357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-82ceb50c-b713-41df-812e-2156581a4937 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:25.365635 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-82ceb50c-b713-41df-812e-2156581a4937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:25.442230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 573/1140] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:25.466269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae492e8e-b799-4cfd-abe2-606e36766e2f None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:25.569496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae492e8e-b799-4cfd-abe2-606e36766e2f None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:25.595530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:25.605411 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad89d4-da48-4847-b4e4-59faffe49a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:25.606197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad89d4-da48-4847-b4e4-59faffe49a59 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:25.606480 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad89d4-da48-4847-b4e4-59faffe49a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:25.606752 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ad89d4-da48-4847-b4e4-59faffe49a59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:25.618535 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ad89d4-da48-4847-b4e4-59faffe49a59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:32:25.625970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-16ad89d4-da48-4847-b4e4-59faffe49a59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:32:25.625970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:25.625970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:25.625970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:25.625970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:25.625970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:25.625970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:25.625970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:25.625970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:25.625970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:25.625970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:25.625970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:25.625970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:32:25.625970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:32:25.625970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:32:25.625970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:32:25.627239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:32:25.627239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:32:25.627239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:32:25.627239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:32:25.627239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:25.627239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:25.627239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:25.627239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:25.627239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:25.627239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:25.627239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:25.627239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:25.627239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:25.627239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:25.627239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:25.627239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:25.628984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:25.628984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:25.628984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:25.628984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:25.628984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:32:25.628984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:32:25.628984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:32:25.628984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:32:25.628984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:32:25.628984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:32:25.628984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:32:25.628984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:32:25.628984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 568/1141] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:32:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:32:25.636456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ceb50c-b713-41df-812e-2156581a4937 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:25.688199 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:25.743534 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ceb50c-b713-41df-812e-2156581a4937 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:25.780545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcab2d-8759-44c4-ac0f-85d7d4828fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:25.781207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcab2d-8759-44c4-ac0f-85d7d4828fea None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:25.781483 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcab2d-8759-44c4-ac0f-85d7d4828fea None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:25.781755 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcab2d-8759-44c4-ac0f-85d7d4828fea None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:25.782491 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dcab2d-8759-44c4-ac0f-85d7d4828fea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:25.782802 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dcab2d-8759-44c4-ac0f-85d7d4828fea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:25.784222 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dcab2d-8759-44c4-ac0f-85d7d4828fea None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:25.792888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 574/1142] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:32:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:25.799050 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad89d4-da48-4847-b4e4-59faffe49a59 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:25.905981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ad89d4-da48-4847-b4e4-59faffe49a59 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:25.942562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd927ef-9fd1-4fb6-82dc-a43cfccb636c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:25.943498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd927ef-9fd1-4fb6-82dc-a43cfccb636c None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:25.943773 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd927ef-9fd1-4fb6-82dc-a43cfccb636c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:25.944048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd927ef-9fd1-4fb6-82dc-a43cfccb636c None admin] PATH_INFO: `/v3/users/dfa9e21943db454f84e6c9d7585a186c` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:25.957366 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd927ef-9fd1-4fb6-82dc-a43cfccb636c None admin] RBAC: Authorizing `identity:get_user(user_id=dfa9e21943db454f84e6c9d7585a186c)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:25.958844 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd927ef-9fd1-4fb6-82dc-a43cfccb636c None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:25.960512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 569/1143] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:32:25 2022] GET /identity/v3/users/dfa9e21943db454f84e6c9d7585a186c => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:25.969568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcab2d-8759-44c4-ac0f-85d7d4828fea None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:26.068800 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcab2d-8759-44c4-ac0f-85d7d4828fea None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:26.075476 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:26.113171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d11a0-e8cb-4b48-b54b-1a7237e8340d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:26.113806 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d11a0-e8cb-4b48-b54b-1a7237e8340d None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:26.114091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d11a0-e8cb-4b48-b54b-1a7237e8340d None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:26.114361 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d11a0-e8cb-4b48-b54b-1a7237e8340d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:26.184658 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d11a0-e8cb-4b48-b54b-1a7237e8340d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:26.186108 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d11a0-e8cb-4b48-b54b-1a7237e8340d None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:26.284865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 575/1144] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:32:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 05:32:26.291486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd927ef-9fd1-4fb6-82dc-a43cfccb636c None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:26.341236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:26.395309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd927ef-9fd1-4fb6-82dc-a43cfccb636c None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:26.430903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c32b43-947b-45a7-b02d-a04f6d64c8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:26.431779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c32b43-947b-45a7-b02d-a04f6d64c8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:26.432060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c32b43-947b-45a7-b02d-a04f6d64c8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:26.432326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c32b43-947b-45a7-b02d-a04f6d64c8f9 None admin] PATH_INFO: `/v3/projects/cddd4b94756e410b8c79153a3c272213` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:26.433526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c32b43-947b-45a7-b02d-a04f6d64c8f9 None admin] RBAC: Authorizing `identity:get_project(project_id=cddd4b94756e410b8c79153a3c272213)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:26.435042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c32b43-947b-45a7-b02d-a04f6d64c8f9 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:26.436679 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 570/1145] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:32:26 2022] GET /identity/v3/projects/cddd4b94756e410b8c79153a3c272213 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:26.446425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d11a0-e8cb-4b48-b54b-1a7237e8340d None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:26.459039 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:26.557905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d11a0-e8cb-4b48-b54b-1a7237e8340d None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:26.592397 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbbe81-ac42-4ca2-b943-c5c8b7760943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:26.593040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbbe81-ac42-4ca2-b943-c5c8b7760943 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:26.593320 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbbe81-ac42-4ca2-b943-c5c8b7760943 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:26.593587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbbe81-ac42-4ca2-b943-c5c8b7760943 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:26.594404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbbe81-ac42-4ca2-b943-c5c8b7760943 None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=cddd4b94756e410b8c79153a3c272213, user.id=dfa9e21943db454f84e6c9d7585a186c) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:26.594706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbbe81-ac42-4ca2-b943-c5c8b7760943 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:26.596175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbbe81-ac42-4ca2-b943-c5c8b7760943 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:26.601671 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 576/1146] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:32:26 2022] GET /identity/v3/role_assignments?user.id=dfa9e21943db454f84e6c9d7585a186c&scope.project.id=cddd4b94756e410b8c79153a3c272213&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:28.592571 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd10b70-94a3-4ca7-83d2-0e3bb3d95c83 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:28.593440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd10b70-94a3-4ca7-83d2-0e3bb3d95c83 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:28.593440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd10b70-94a3-4ca7-83d2-0e3bb3d95c83 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:28.595723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 571/1147] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:28.601083 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226ec3a-6dd9-4ba6-b66a-7882e16a7a9d None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:28.601498 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226ec3a-6dd9-4ba6-b66a-7882e16a7a9d None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:28.601608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226ec3a-6dd9-4ba6-b66a-7882e16a7a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:28.619796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-9226ec3a-6dd9-4ba6-b66a-7882e16a7a9d None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:28.620956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9226ec3a-6dd9-4ba6-b66a-7882e16a7a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:28.690863 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:28.696364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 577/1148] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:28.719548 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd10b70-94a3-4ca7-83d2-0e3bb3d95c83 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:28.827372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd10b70-94a3-4ca7-83d2-0e3bb3d95c83 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:28.841740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:28.864777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca536e5-0c13-4e75-ade0-eba715a823e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:28.865533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca536e5-0c13-4e75-ade0-eba715a823e6 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:28.865808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca536e5-0c13-4e75-ade0-eba715a823e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:28.866101 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca536e5-0c13-4e75-ade0-eba715a823e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:28.876581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca536e5-0c13-4e75-ade0-eba715a823e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:32:28.884060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-6ca536e5-0c13-4e75-ade0-eba715a823e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:32:28.884060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:28.884060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:28.884060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:28.884060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:28.884060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:28.884060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:28.884060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:28.884060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:28.884060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:28.884060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:28.884060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:28.884060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:32:28.884060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:32:28.884060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:32:28.884060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:32:28.885171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:32:28.885171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:32:28.885171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:32:28.885171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:32:28.885171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:28.885171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:28.885171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:28.885171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:28.885171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:28.885171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:28.885171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:28.885171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:28.885171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:28.885171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:28.885171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:28.885171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:28.886258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:28.886258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:28.886258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:28.886258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:28.886258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:32:28.886258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:32:28.886258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:32:28.886258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:32:28.886258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:32:28.886258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:32:28.886258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:32:28.886258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:32:28.886258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 572/1149] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:32:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:32:28.891692 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226ec3a-6dd9-4ba6-b66a-7882e16a7a9d None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:28.996925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226ec3a-6dd9-4ba6-b66a-7882e16a7a9d None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:29.032008 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378ccf5-3484-4f47-a583-cfcce0e96104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:29.032666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378ccf5-3484-4f47-a583-cfcce0e96104 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:29.032947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378ccf5-3484-4f47-a583-cfcce0e96104 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:29.033219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378ccf5-3484-4f47-a583-cfcce0e96104 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:29.033952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378ccf5-3484-4f47-a583-cfcce0e96104 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:29.034261 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378ccf5-3484-4f47-a583-cfcce0e96104 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:29.035753 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378ccf5-3484-4f47-a583-cfcce0e96104 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:29.044468 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 578/1150] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:32:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:29.051485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca536e5-0c13-4e75-ade0-eba715a823e6 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:29.156369 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca536e5-0c13-4e75-ade0-eba715a823e6 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:29.190370 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e7403-b33e-47f0-b801-0bad20fa55a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:29.191249 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e7403-b33e-47f0-b801-0bad20fa55a1 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:29.191524 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e7403-b33e-47f0-b801-0bad20fa55a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:29.191795 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e7403-b33e-47f0-b801-0bad20fa55a1 None admin] PATH_INFO: `/v3/users/dfa9e21943db454f84e6c9d7585a186c` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:29.193168 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e7403-b33e-47f0-b801-0bad20fa55a1 None admin] RBAC: Authorizing `identity:get_user(user_id=dfa9e21943db454f84e6c9d7585a186c)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:29.194588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e7403-b33e-47f0-b801-0bad20fa55a1 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:29.196209 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 573/1151] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:32:29 2022] GET /identity/v3/users/dfa9e21943db454f84e6c9d7585a186c => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:29.202159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378ccf5-3484-4f47-a583-cfcce0e96104 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:29.267174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:29.308308 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378ccf5-3484-4f47-a583-cfcce0e96104 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:29.346515 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c520e6a-6454-48fd-9b81-c5bdb1f836b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:29.347403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c520e6a-6454-48fd-9b81-c5bdb1f836b9 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:29.347682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c520e6a-6454-48fd-9b81-c5bdb1f836b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:29.347954 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c520e6a-6454-48fd-9b81-c5bdb1f836b9 None admin] PATH_INFO: `/v3/projects/cddd4b94756e410b8c79153a3c272213` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:29.349163 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c520e6a-6454-48fd-9b81-c5bdb1f836b9 None admin] RBAC: Authorizing `identity:get_project(project_id=cddd4b94756e410b8c79153a3c272213)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:29.350597 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c520e6a-6454-48fd-9b81-c5bdb1f836b9 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:29.352325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 579/1152] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:32:29 2022] GET /identity/v3/projects/cddd4b94756e410b8c79153a3c272213 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:29.361078 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e7403-b33e-47f0-b801-0bad20fa55a1 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:29.403108 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:29.462019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e7403-b33e-47f0-b801-0bad20fa55a1 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:29.496451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a594619-a003-4ff5-a4bf-3eca3d7c977a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:29.497324 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a594619-a003-4ff5-a4bf-3eca3d7c977a None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:29.497603 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a594619-a003-4ff5-a4bf-3eca3d7c977a None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:29.497874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a594619-a003-4ff5-a4bf-3eca3d7c977a None admin] PATH_INFO: `/v3/projects/cddd4b94756e410b8c79153a3c272213/users/dfa9e21943db454f84e6c9d7585a186c/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:29.500102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a594619-a003-4ff5-a4bf-3eca3d7c977a None admin] RBAC: Authorizing `identity:create_grant(project_id=cddd4b94756e410b8c79153a3c272213, user_id=dfa9e21943db454f84e6c9d7585a186c, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:29.501522 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a594619-a003-4ff5-a4bf-3eca3d7c977a None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:29.518021 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 574/1153] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:32:29 2022] PUT /identity/v3/projects/cddd4b94756e410b8c79153a3c272213/users/dfa9e21943db454f84e6c9d7585a186c/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:32:31.490579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5ece2-672e-4c35-be68-8344ce14ccd1 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:31.491388 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5ece2-672e-4c35-be68-8344ce14ccd1 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:31.491388 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5ece2-672e-4c35-be68-8344ce14ccd1 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:31.493616 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 580/1154] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:31.500024 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f8968-5784-4d64-89f0-fb2d2b5bf80a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:31.500781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f8968-5784-4d64-89f0-fb2d2b5bf80a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:31.500959 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f8968-5784-4d64-89f0-fb2d2b5bf80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:31.520189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-895f8968-5784-4d64-89f0-fb2d2b5bf80a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:31.521415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-895f8968-5784-4d64-89f0-fb2d2b5bf80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:31.615714 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 575/1155] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:31.651318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5ece2-672e-4c35-be68-8344ce14ccd1 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:31.769217 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5ece2-672e-4c35-be68-8344ce14ccd1 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:31.797702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:31.804812 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7fd80-cbc6-4fb7-bfad-7e4e9ea263eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:31.805623 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7fd80-cbc6-4fb7-bfad-7e4e9ea263eb None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:31.805904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7fd80-cbc6-4fb7-bfad-7e4e9ea263eb None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:31.806174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7fd80-cbc6-4fb7-bfad-7e4e9ea263eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:31.817330 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7fd80-cbc6-4fb7-bfad-7e4e9ea263eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:32:31.824882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-2cf7fd80-cbc6-4fb7-bfad-7e4e9ea263eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:32:31.824882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:31.824882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:31.824882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:31.824882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:31.824882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:31.824882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:31.824882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:31.824882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:31.824882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:31.824882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:31.824882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:31.824882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:32:31.824882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:32:31.824882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:32:31.824882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:32:31.826012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:32:31.826012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:32:31.826012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:32:31.826012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:32:31.826012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:31.826012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:31.826012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:31.826012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:31.826012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:31.826012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:31.826012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:31.826012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:31.826012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:31.826012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:31.826012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:31.826012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:31.834579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:31.834579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:31.834579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:31.834579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:31.834579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:32:31.834579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:32:31.834579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:32:31.834579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:32:31.834579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:32:31.834579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:32:31.834579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:32:31.834579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:32:31.834579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 581/1156] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:32:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:32:31.835474 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f8968-5784-4d64-89f0-fb2d2b5bf80a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:31.880340 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:31.957718 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f8968-5784-4d64-89f0-fb2d2b5bf80a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:31.996803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9dc79-3724-4735-9683-366ae8e76ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:31.997804 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9dc79-3724-4735-9683-366ae8e76ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:31.998242 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9dc79-3724-4735-9683-366ae8e76ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:31.998656 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9dc79-3724-4735-9683-366ae8e76ef8 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:31.999898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9dc79-3724-4735-9683-366ae8e76ef8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:32.000364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9dc79-3724-4735-9683-366ae8e76ef8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:32.002553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9dc79-3724-4735-9683-366ae8e76ef8 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:32.012224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 576/1157] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:32:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:32.018275 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7fd80-cbc6-4fb7-bfad-7e4e9ea263eb None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:32.130590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7fd80-cbc6-4fb7-bfad-7e4e9ea263eb None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:32.166350 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5bfa5-581a-48c6-90a9-8629385be964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:32.167242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5bfa5-581a-48c6-90a9-8629385be964 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:32.167523 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5bfa5-581a-48c6-90a9-8629385be964 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:32.167795 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5bfa5-581a-48c6-90a9-8629385be964 None admin] PATH_INFO: `/v3/users/dfa9e21943db454f84e6c9d7585a186c` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:32.169100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5bfa5-581a-48c6-90a9-8629385be964 None admin] RBAC: Authorizing `identity:get_user(user_id=dfa9e21943db454f84e6c9d7585a186c)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:32.170536 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc5bfa5-581a-48c6-90a9-8629385be964 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:32.172183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 582/1158] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:32:32 2022] GET /identity/v3/users/dfa9e21943db454f84e6c9d7585a186c => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:32.178926 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9dc79-3724-4735-9683-366ae8e76ef8 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:32.281648 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:32.292137 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9dc79-3724-4735-9683-366ae8e76ef8 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:32.327309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ea385-3eb7-4a61-8693-99a62a3cfc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:32.327960 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ea385-3eb7-4a61-8693-99a62a3cfc3e None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:32.328242 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ea385-3eb7-4a61-8693-99a62a3cfc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:32.328521 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ea385-3eb7-4a61-8693-99a62a3cfc3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:32.402451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ea385-3eb7-4a61-8693-99a62a3cfc3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:32.403873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ea385-3eb7-4a61-8693-99a62a3cfc3e None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:32.500871 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:32.517279 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 577/1159] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:32:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 05:32:32.524769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5bfa5-581a-48c6-90a9-8629385be964 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:32.584537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:32.630683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5bfa5-581a-48c6-90a9-8629385be964 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:32.665826 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671942c-d731-4522-ac2d-7bf5dba704dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:32.666706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671942c-d731-4522-ac2d-7bf5dba704dd None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:32.666983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671942c-d731-4522-ac2d-7bf5dba704dd None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:32.667286 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671942c-d731-4522-ac2d-7bf5dba704dd None admin] PATH_INFO: `/v3/projects/cddd4b94756e410b8c79153a3c272213` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:32.668461 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671942c-d731-4522-ac2d-7bf5dba704dd None admin] RBAC: Authorizing `identity:get_project(project_id=cddd4b94756e410b8c79153a3c272213)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:32.669896 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671942c-d731-4522-ac2d-7bf5dba704dd None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:32.671634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 583/1160] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:32:32 2022] GET /identity/v3/projects/cddd4b94756e410b8c79153a3c272213 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:32.680050 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ea385-3eb7-4a61-8693-99a62a3cfc3e None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:32.779263 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ea385-3eb7-4a61-8693-99a62a3cfc3e None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:32.814633 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281e3ba-2d83-428d-af3d-06e827146a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:32.815319 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281e3ba-2d83-428d-af3d-06e827146a7e None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:32.815606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281e3ba-2d83-428d-af3d-06e827146a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:32.815880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281e3ba-2d83-428d-af3d-06e827146a7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:32.816711 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281e3ba-2d83-428d-af3d-06e827146a7e None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=cddd4b94756e410b8c79153a3c272213, user.id=dfa9e21943db454f84e6c9d7585a186c) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:32.817024 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281e3ba-2d83-428d-af3d-06e827146a7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:32.818561 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d281e3ba-2d83-428d-af3d-06e827146a7e None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:32.824548 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 578/1161] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:32:32 2022] GET /identity/v3/role_assignments?user.id=dfa9e21943db454f84e6c9d7585a186c&scope.project.id=cddd4b94756e410b8c79153a3c272213&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:34.837991 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73b1c8-5f25-4a4b-bd85-3e59da5b4546 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:34.838576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73b1c8-5f25-4a4b-bd85-3e59da5b4546 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:34.838576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73b1c8-5f25-4a4b-bd85-3e59da5b4546 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:34.839930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 584/1162] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:34.846300 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5e535-e54c-416d-95d0-0d3ac73b8404 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:34.847060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5e535-e54c-416d-95d0-0d3ac73b8404 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:34.847178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5e535-e54c-416d-95d0-0d3ac73b8404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:34.871325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-49d5e535-e54c-416d-95d0-0d3ac73b8404 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:34.872514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-49d5e535-e54c-416d-95d0-0d3ac73b8404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:34.925875 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:34.944276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 579/1163] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:34.966934 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73b1c8-5f25-4a4b-bd85-3e59da5b4546 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:35.076974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73b1c8-5f25-4a4b-bd85-3e59da5b4546 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:35.102694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:35.112577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abbee0-ee8d-44c3-9e04-60b29de9e33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:35.113228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abbee0-ee8d-44c3-9e04-60b29de9e33d None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:35.113513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abbee0-ee8d-44c3-9e04-60b29de9e33d None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:35.113789 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abbee0-ee8d-44c3-9e04-60b29de9e33d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:35.181860 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1abbee0-ee8d-44c3-9e04-60b29de9e33d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:35.183285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1abbee0-ee8d-44c3-9e04-60b29de9e33d None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:35.284920 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 585/1164] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:32:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 05:32:35.292182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5e535-e54c-416d-95d0-0d3ac73b8404 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:35.408121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5e535-e54c-416d-95d0-0d3ac73b8404 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:35.444402 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121567b-33d6-4a5a-8a3f-7c3f28295396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:35.445153 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121567b-33d6-4a5a-8a3f-7c3f28295396 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:35.445433 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121567b-33d6-4a5a-8a3f-7c3f28295396 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:35.445708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121567b-33d6-4a5a-8a3f-7c3f28295396 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:35.453683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121567b-33d6-4a5a-8a3f-7c3f28295396 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:35.455182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121567b-33d6-4a5a-8a3f-7c3f28295396 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:35.470432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-d121567b-33d6-4a5a-8a3f-7c3f28295396 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 05:32:35.470432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:35.470432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 21 05:32:35.470432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 21 05:32:35.470432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:32:35.470432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:32:35.470432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:32:35.470432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:32:35.470432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 21 05:32:35.470432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:32:35.470432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:32:35.470432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:32:35.470432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:35.470432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:35.470432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:35.470432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:35.470432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:35.471642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:35.471642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:35.471642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:35.471642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:35.471642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:35.471642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:35.471642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:32:35.471642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:32:35.471642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:32:35.471642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:32:35.471642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:35.471642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:35.471642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:35.471642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:35.471642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:35.471642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:35.472504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:35.472504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:35.472504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:35.472504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:35.472504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:35.472504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:35.472504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:35.472504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:35.472504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:35.472504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:35.472504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 21 05:32:35.472504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:32:35.472504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 21 05:32:35.472504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:32:35.472504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 580/1165] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jul 21 05:32:35 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:32:35.477962 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abbee0-ee8d-44c3-9e04-60b29de9e33d None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:35.505379 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:35.582029 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abbee0-ee8d-44c3-9e04-60b29de9e33d None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:35.618298 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a6515-c521-48bb-a696-7ef8dd3d1ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:35.618944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a6515-c521-48bb-a696-7ef8dd3d1ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:35.619261 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a6515-c521-48bb-a696-7ef8dd3d1ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:35.619534 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a6515-c521-48bb-a696-7ef8dd3d1ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:35.620254 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a6515-c521-48bb-a696-7ef8dd3d1ec9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:35.620568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a6515-c521-48bb-a696-7ef8dd3d1ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:35.621930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a6515-c521-48bb-a696-7ef8dd3d1ec9 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:35.629123 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 586/1166] 10.222.0.44 () {56 vars in 1115 bytes} [Thu Jul 21 05:32:35 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:37.622740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278c9bb-46f1-43ab-aa84-26eea963085a None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:37.623384 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278c9bb-46f1-43ab-aa84-26eea963085a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:37.623384 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278c9bb-46f1-43ab-aa84-26eea963085a None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:37.624845 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 581/1167] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:37.631374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57055859-f735-4e21-a751-bda48c46e5a2 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:37.632025 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57055859-f735-4e21-a751-bda48c46e5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:37.632399 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57055859-f735-4e21-a751-bda48c46e5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:37.655510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-57055859-f735-4e21-a751-bda48c46e5a2 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:37.656697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-57055859-f735-4e21-a751-bda48c46e5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:37.725980 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 587/1168] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:37.748818 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278c9bb-46f1-43ab-aa84-26eea963085a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:37.810424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:37.875164 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278c9bb-46f1-43ab-aa84-26eea963085a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:37.911472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611d39b-5352-4b6b-8d0c-413526c6e325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:37.912492 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611d39b-5352-4b6b-8d0c-413526c6e325 None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:37.912772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611d39b-5352-4b6b-8d0c-413526c6e325 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:37.913035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611d39b-5352-4b6b-8d0c-413526c6e325 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:37.913800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611d39b-5352-4b6b-8d0c-413526c6e325 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:37.915231 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611d39b-5352-4b6b-8d0c-413526c6e325 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:37.934044 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 582/1169] 10.222.0.44 () {60 vars in 1138 bytes} [Thu Jul 21 05:32:37 2022] POST /identity/v3/domains => generated 267 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:32:39.976451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862cefa-6709-4909-948d-8cb603ec06ed None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:39.977336 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862cefa-6709-4909-948d-8cb603ec06ed None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:39.977336 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862cefa-6709-4909-948d-8cb603ec06ed None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:39.979525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 588/1170] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:39.984903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06858d8-ef54-46c7-b602-65f0031c6541 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:39.985344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06858d8-ef54-46c7-b602-65f0031c6541 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:39.985445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06858d8-ef54-46c7-b602-65f0031c6541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:40.003145 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-e06858d8-ef54-46c7-b602-65f0031c6541 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:40.004326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e06858d8-ef54-46c7-b602-65f0031c6541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:40.076199 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 583/1171] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:40.100195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862cefa-6709-4909-948d-8cb603ec06ed None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:40.127245 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:40.211055 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862cefa-6709-4909-948d-8cb603ec06ed None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:40.245367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a799ce-4d28-4f44-ab32-12cf4cc5ac34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:40.246155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a799ce-4d28-4f44-ab32-12cf4cc5ac34 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:40.246436 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a799ce-4d28-4f44-ab32-12cf4cc5ac34 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:40.246708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a799ce-4d28-4f44-ab32-12cf4cc5ac34 None admin] PATH_INFO: `/v3/domains/dfeb9543d9be41f3ba3b6e44c77088d5` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:40.257635 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a799ce-4d28-4f44-ab32-12cf4cc5ac34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=dfeb9543d9be41f3ba3b6e44c77088d5)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:40.259131 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a799ce-4d28-4f44-ab32-12cf4cc5ac34 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:40.260761 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 589/1172] 10.222.0.44 () {56 vars in 1183 bytes} [Thu Jul 21 05:32:40 2022] GET /identity/v3/domains/dfeb9543d9be41f3ba3b6e44c77088d5 => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:40.269918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06858d8-ef54-46c7-b602-65f0031c6541 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:40.312250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:40.393873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06858d8-ef54-46c7-b602-65f0031c6541 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:40.433117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17f723-6a0f-4849-b314-3dbeadc23ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:40.433785 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17f723-6a0f-4849-b314-3dbeadc23ca2 None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:40.434070 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17f723-6a0f-4849-b314-3dbeadc23ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:40.434341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17f723-6a0f-4849-b314-3dbeadc23ca2 None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:40.435058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17f723-6a0f-4849-b314-3dbeadc23ca2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:40.436587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17f723-6a0f-4849-b314-3dbeadc23ca2 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:40.465187 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 584/1173] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jul 21 05:32:40 2022] POST /identity/v3/projects => generated 361 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:32:42.521116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e507284-e012-4d47-95a1-a06c2946bf2a None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:42.522007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e507284-e012-4d47-95a1-a06c2946bf2a None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:42.522007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e507284-e012-4d47-95a1-a06c2946bf2a None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:42.524243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 590/1174] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:42.530625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bafd52-7985-45f9-ba97-cace986397d0 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:42.531440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bafd52-7985-45f9-ba97-cace986397d0 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:42.531602 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bafd52-7985-45f9-ba97-cace986397d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:42.551606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-87bafd52-7985-45f9-ba97-cace986397d0 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:42.552788 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-87bafd52-7985-45f9-ba97-cace986397d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:42.623003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 585/1175] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:42.648123 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e507284-e012-4d47-95a1-a06c2946bf2a None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:42.736242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:42.764437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e507284-e012-4d47-95a1-a06c2946bf2a None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:42.799741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220afb7c-1763-441e-9828-4b9e9eee0e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:42.800549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220afb7c-1763-441e-9828-4b9e9eee0e37 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:42.800831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220afb7c-1763-441e-9828-4b9e9eee0e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:42.801108 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220afb7c-1763-441e-9828-4b9e9eee0e37 None admin] PATH_INFO: `/v3/domains/dfeb9543d9be41f3ba3b6e44c77088d5` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:42.802408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220afb7c-1763-441e-9828-4b9e9eee0e37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=dfeb9543d9be41f3ba3b6e44c77088d5)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:42.803896 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220afb7c-1763-441e-9828-4b9e9eee0e37 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:42.805525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 591/1176] 10.222.0.44 () {56 vars in 1183 bytes} [Thu Jul 21 05:32:42 2022] GET /identity/v3/domains/dfeb9543d9be41f3ba3b6e44c77088d5 => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:42.811558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bafd52-7985-45f9-ba97-cace986397d0 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:42.831910 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:42.922698 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bafd52-7985-45f9-ba97-cace986397d0 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:42.958534 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76843b-c639-4d15-b080-203cc6becb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:42.959235 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76843b-c639-4d15-b080-203cc6becb7f None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:42.959517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76843b-c639-4d15-b080-203cc6becb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:42.959792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76843b-c639-4d15-b080-203cc6becb7f None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:42.960507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76843b-c639-4d15-b080-203cc6becb7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:42.961954 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76843b-c639-4d15-b080-203cc6becb7f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:42.983795 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 586/1177] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:32:42 2022] POST /identity/v3/users => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:32:45.006268 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f968699-4aef-41f9-8eb2-af1b0fa3edbf None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:45.006797 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f968699-4aef-41f9-8eb2-af1b0fa3edbf None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:45.006797 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f968699-4aef-41f9-8eb2-af1b0fa3edbf None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:45.008133 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 592/1178] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:45.014520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80b2d4-8ba6-463e-b76d-bb6f7e6d70c5 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:45.015322 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80b2d4-8ba6-463e-b76d-bb6f7e6d70c5 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:45.015450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80b2d4-8ba6-463e-b76d-bb6f7e6d70c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:45.040635 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-6d80b2d4-8ba6-463e-b76d-bb6f7e6d70c5 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:45.041827 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6d80b2d4-8ba6-463e-b76d-bb6f7e6d70c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:45.112622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 587/1179] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:45.136031 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f968699-4aef-41f9-8eb2-af1b0fa3edbf None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:45.243002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f968699-4aef-41f9-8eb2-af1b0fa3edbf None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:45.277198 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ec77c-c5fa-42b5-a510-650147428a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:45.277915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ec77c-c5fa-42b5-a510-650147428a65 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:45.278191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ec77c-c5fa-42b5-a510-650147428a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:45.278464 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ec77c-c5fa-42b5-a510-650147428a65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:45.289785 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ec77c-c5fa-42b5-a510-650147428a65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:32:45.297170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-951ec77c-c5fa-42b5-a510-650147428a65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:32:45.297170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:45.297170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:45.297170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:45.297170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:45.297170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:45.297170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:45.297170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:45.297170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:45.297170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:45.297170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:45.297170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:45.297170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:32:45.297170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:32:45.297170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:32:45.297170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:32:45.298312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:32:45.298312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:32:45.298312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:32:45.298312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:32:45.298312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:45.298312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:45.298312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:45.298312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:45.298312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:45.298312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:45.298312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:45.298312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:45.298312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:45.298312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:45.298312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:45.298312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:45.299686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:45.299686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:45.299686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:45.299686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:45.299686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:32:45.299686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:32:45.299686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:32:45.299686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:32:45.299686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:32:45.299686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:32:45.299686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:32:45.299686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:32:45.299686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 593/1180] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:32:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:32:45.304994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80b2d4-8ba6-463e-b76d-bb6f7e6d70c5 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:45.309682 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:45.413785 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80b2d4-8ba6-463e-b76d-bb6f7e6d70c5 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:45.451674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea33e9-2b72-4016-bd38-99d3d000e83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:45.452341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea33e9-2b72-4016-bd38-99d3d000e83e None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:45.452625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea33e9-2b72-4016-bd38-99d3d000e83e None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:45.452899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea33e9-2b72-4016-bd38-99d3d000e83e None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:45.453652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea33e9-2b72-4016-bd38-99d3d000e83e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:45.453963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea33e9-2b72-4016-bd38-99d3d000e83e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:45.455416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea33e9-2b72-4016-bd38-99d3d000e83e None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:45.463759 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 588/1181] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:32:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:45.469612 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ec77c-c5fa-42b5-a510-650147428a65 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:45.480864 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:45.575284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ec77c-c5fa-42b5-a510-650147428a65 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:45.611553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d8758-827e-41d7-8fd5-eab835dbb5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:45.612415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d8758-827e-41d7-8fd5-eab835dbb5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:45.612701 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d8758-827e-41d7-8fd5-eab835dbb5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:45.612979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d8758-827e-41d7-8fd5-eab835dbb5c6 None admin] PATH_INFO: `/v3/users/5999dd8c7ba444bb898caad572d4cff1` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:45.627773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d8758-827e-41d7-8fd5-eab835dbb5c6 None admin] RBAC: Authorizing `identity:get_user(user_id=5999dd8c7ba444bb898caad572d4cff1)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:45.629227 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d8758-827e-41d7-8fd5-eab835dbb5c6 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:45.630937 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 594/1182] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:32:45 2022] GET /identity/v3/users/5999dd8c7ba444bb898caad572d4cff1 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:45.639498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea33e9-2b72-4016-bd38-99d3d000e83e None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:45.710061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:45.750883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea33e9-2b72-4016-bd38-99d3d000e83e None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:45.789629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e7200-475c-4b72-af94-3ac4e3eca6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:45.790329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e7200-475c-4b72-af94-3ac4e3eca6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:45.790611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e7200-475c-4b72-af94-3ac4e3eca6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:45.790882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e7200-475c-4b72-af94-3ac4e3eca6c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:45.866713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e7200-475c-4b72-af94-3ac4e3eca6c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:45.868117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e7200-475c-4b72-af94-3ac4e3eca6c3 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:45.931839 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:45.970378 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 589/1183] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:32:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 05:32:45.976588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d8758-827e-41d7-8fd5-eab835dbb5c6 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:46.042750 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:46.081116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d8758-827e-41d7-8fd5-eab835dbb5c6 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:46.117708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8740d-6ade-4055-9498-156f1f1b2ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:46.118496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8740d-6ade-4055-9498-156f1f1b2ccc None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:46.118769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8740d-6ade-4055-9498-156f1f1b2ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:46.119038 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8740d-6ade-4055-9498-156f1f1b2ccc None admin] PATH_INFO: `/v3/projects/e6350a5e211643c681dd82bada61f3c0` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:46.120357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8740d-6ade-4055-9498-156f1f1b2ccc None admin] RBAC: Authorizing `identity:get_project(project_id=e6350a5e211643c681dd82bada61f3c0)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:46.121798 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8740d-6ade-4055-9498-156f1f1b2ccc None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:46.123531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 595/1184] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:32:45 2022] GET /identity/v3/projects/e6350a5e211643c681dd82bada61f3c0 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:46.132481 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e7200-475c-4b72-af94-3ac4e3eca6c3 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:46.236731 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e7200-475c-4b72-af94-3ac4e3eca6c3 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:46.272194 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b55bc-c6bf-4784-bc1d-c5f1475533c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:46.272837 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b55bc-c6bf-4784-bc1d-c5f1475533c9 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:46.273111 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b55bc-c6bf-4784-bc1d-c5f1475533c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:46.273377 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b55bc-c6bf-4784-bc1d-c5f1475533c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:46.274207 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b55bc-c6bf-4784-bc1d-c5f1475533c9 None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=e6350a5e211643c681dd82bada61f3c0, user.id=5999dd8c7ba444bb898caad572d4cff1) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:46.274515 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b55bc-c6bf-4784-bc1d-c5f1475533c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:46.275987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b55bc-c6bf-4784-bc1d-c5f1475533c9 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:46.281579 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 590/1185] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:32:46 2022] GET /identity/v3/role_assignments?user.id=5999dd8c7ba444bb898caad572d4cff1&scope.project.id=e6350a5e211643c681dd82bada61f3c0&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:32:48.256275 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f424736-2d26-4206-8202-075084ef2fcc None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:48.257171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f424736-2d26-4206-8202-075084ef2fcc None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:48.257171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f424736-2d26-4206-8202-075084ef2fcc None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:48.259369 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 596/1186] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:48.265688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a0578-056a-48de-9776-652561df853c None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:48.266114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a0578-056a-48de-9776-652561df853c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:48.266256 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a0578-056a-48de-9776-652561df853c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:48.286260 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-f89a0578-056a-48de-9776-652561df853c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:48.287843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f89a0578-056a-48de-9776-652561df853c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:48.313898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:48.365687 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 591/1187] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:48.389556 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f424736-2d26-4206-8202-075084ef2fcc None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:48.504807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f424736-2d26-4206-8202-075084ef2fcc None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:48.539966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36f782-798a-46ab-9606-91f0e265e2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:48.540693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36f782-798a-46ab-9606-91f0e265e2dd None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:48.540972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36f782-798a-46ab-9606-91f0e265e2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:48.541238 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36f782-798a-46ab-9606-91f0e265e2dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:48.543365 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:48.552636 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f36f782-798a-46ab-9606-91f0e265e2dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:32:48.560179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-1f36f782-798a-46ab-9606-91f0e265e2dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:32:48.560179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:48.560179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:48.560179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:48.560179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:48.560179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:48.560179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:48.560179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:48.560179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:48.560179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:48.560179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:48.560179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:48.560179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:32:48.560179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:32:48.560179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:32:48.560179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:32:48.561256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:32:48.561256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:32:48.561256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:32:48.561256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:32:48.561256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:48.561256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:48.561256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:48.561256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:48.561256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:48.561256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:48.561256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:48.561256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:48.561256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:48.561256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:48.561256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:48.561256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:48.562314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:48.562314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:48.562314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:48.562314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:48.562314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:32:48.562314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:32:48.562314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:32:48.562314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:32:48.562314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:32:48.562314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:32:48.562314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:32:48.562314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:32:48.562314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 597/1188] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:32:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:32:48.567688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a0578-056a-48de-9776-652561df853c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:48.676169 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a0578-056a-48de-9776-652561df853c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:48.713143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58e6e1-164b-4965-a89c-42b3705b1cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:48.713841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58e6e1-164b-4965-a89c-42b3705b1cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:48.714123 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58e6e1-164b-4965-a89c-42b3705b1cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:48.714396 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58e6e1-164b-4965-a89c-42b3705b1cb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:48.715173 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58e6e1-164b-4965-a89c-42b3705b1cb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:48.715485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58e6e1-164b-4965-a89c-42b3705b1cb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:48.716862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58e6e1-164b-4965-a89c-42b3705b1cb7 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:48.725291 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 592/1189] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:32:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:48.731524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36f782-798a-46ab-9606-91f0e265e2dd None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:48.848009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36f782-798a-46ab-9606-91f0e265e2dd None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:48.883144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de9887-b3cb-4138-9efc-bfd334a3de2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:48.884026 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de9887-b3cb-4138-9efc-bfd334a3de2a None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:48.884313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de9887-b3cb-4138-9efc-bfd334a3de2a None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:48.884583 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de9887-b3cb-4138-9efc-bfd334a3de2a None admin] PATH_INFO: `/v3/users/5999dd8c7ba444bb898caad572d4cff1` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:48.885889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de9887-b3cb-4138-9efc-bfd334a3de2a None admin] RBAC: Authorizing `identity:get_user(user_id=5999dd8c7ba444bb898caad572d4cff1)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:48.887370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de9887-b3cb-4138-9efc-bfd334a3de2a None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:48.889038 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 598/1190] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:32:48 2022] GET /identity/v3/users/5999dd8c7ba444bb898caad572d4cff1 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:48.895460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58e6e1-164b-4965-a89c-42b3705b1cb7 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:48.924996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:49.014116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58e6e1-164b-4965-a89c-42b3705b1cb7 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:49.048872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110fa56-0309-44a6-9f49-6352430bf642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:49.049725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110fa56-0309-44a6-9f49-6352430bf642 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:49.050007 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110fa56-0309-44a6-9f49-6352430bf642 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:49.050281 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110fa56-0309-44a6-9f49-6352430bf642 None admin] PATH_INFO: `/v3/projects/e6350a5e211643c681dd82bada61f3c0` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:49.051618 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110fa56-0309-44a6-9f49-6352430bf642 None admin] RBAC: Authorizing `identity:get_project(project_id=e6350a5e211643c681dd82bada61f3c0)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:49.053064 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110fa56-0309-44a6-9f49-6352430bf642 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:49.054764 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 593/1191] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:32:48 2022] GET /identity/v3/projects/e6350a5e211643c681dd82bada61f3c0 => generated 361 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:49.064272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de9887-b3cb-4138-9efc-bfd334a3de2a None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:49.148169 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:49.189355 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de9887-b3cb-4138-9efc-bfd334a3de2a None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:49.223769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901da9ae-a69e-45de-9024-50117a85c2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:49.224653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901da9ae-a69e-45de-9024-50117a85c2b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:49.224934 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901da9ae-a69e-45de-9024-50117a85c2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:49.225203 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901da9ae-a69e-45de-9024-50117a85c2b5 None admin] PATH_INFO: `/v3/projects/e6350a5e211643c681dd82bada61f3c0/users/5999dd8c7ba444bb898caad572d4cff1/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:49.227439 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901da9ae-a69e-45de-9024-50117a85c2b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e6350a5e211643c681dd82bada61f3c0, user_id=5999dd8c7ba444bb898caad572d4cff1, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:49.228867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901da9ae-a69e-45de-9024-50117a85c2b5 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:49.244754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 599/1192] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:32:49 2022] PUT /identity/v3/projects/e6350a5e211643c681dd82bada61f3c0/users/5999dd8c7ba444bb898caad572d4cff1/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:32:51.205436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d7699-d3b4-46b9-a9f2-d3dc1bc24f68 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:51.206314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d7699-d3b4-46b9-a9f2-d3dc1bc24f68 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:51.206314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d7699-d3b4-46b9-a9f2-d3dc1bc24f68 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:51.208323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 594/1193] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:32:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:32:51.214526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba69fe-42fa-4bcd-98c1-592cb9b78652 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:51.215260 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba69fe-42fa-4bcd-98c1-592cb9b78652 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:51.215373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba69fe-42fa-4bcd-98c1-592cb9b78652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:51.241415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-cbba69fe-42fa-4bcd-98c1-592cb9b78652 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:32:51.242603 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-cbba69fe-42fa-4bcd-98c1-592cb9b78652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:32:51.311661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 600/1194] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:32:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:32:51.343070 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d7699-d3b4-46b9-a9f2-d3dc1bc24f68 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:51.441608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:51.453108 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d7699-d3b4-46b9-a9f2-d3dc1bc24f68 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:51.489318 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5126abf-f712-42b8-b13b-bdb9dea28e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:51.490053 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5126abf-f712-42b8-b13b-bdb9dea28e4f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:51.490336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5126abf-f712-42b8-b13b-bdb9dea28e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:51.490609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5126abf-f712-42b8-b13b-bdb9dea28e4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:51.501265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5126abf-f712-42b8-b13b-bdb9dea28e4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:32:51.508823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-c5126abf-f712-42b8-b13b-bdb9dea28e4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:32:51.508823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:32:51.508823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:32:51.508823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:32:51.508823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:32:51.508823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:32:51.508823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:32:51.508823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:32:51.508823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:32:51.508823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:32:51.508823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:32:51.508823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:32:51.508823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:32:51.508823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:32:51.508823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:32:51.508823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:32:51.509969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:32:51.509969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:32:51.509969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:32:51.509969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:32:51.509969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:32:51.509969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:32:51.509969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:32:51.509969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:32:51.509969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:32:51.509969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:32:51.509969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:32:51.509969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:32:51.509969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:32:51.509969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:32:51.509969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:32:51.509969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:32:51.511777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:32:51.511777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:32:51.511777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:32:51.511777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:32:51.511777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:32:51.511777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:32:51.511777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:32:51.511777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:32:51.511777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:32:51.511777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:32:51.511777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:32:51.511777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:32:51.511777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 595/1195] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:32:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:32:51.519255 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba69fe-42fa-4bcd-98c1-592cb9b78652 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:51.571001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:51.622298 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba69fe-42fa-4bcd-98c1-592cb9b78652 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:51.658600 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aae60e-f120-4e4c-9bd6-16db5f7675be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:51.659398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aae60e-f120-4e4c-9bd6-16db5f7675be None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:51.659679 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aae60e-f120-4e4c-9bd6-16db5f7675be None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:51.659954 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aae60e-f120-4e4c-9bd6-16db5f7675be None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:51.660697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aae60e-f120-4e4c-9bd6-16db5f7675be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:51.661087 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aae60e-f120-4e4c-9bd6-16db5f7675be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:51.662818 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aae60e-f120-4e4c-9bd6-16db5f7675be None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:51.671430 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 601/1196] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:32:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:51.678511 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5126abf-f712-42b8-b13b-bdb9dea28e4f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:51.780993 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5126abf-f712-42b8-b13b-bdb9dea28e4f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:51.814339 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bae5e-42d4-4112-bf44-67b78e238464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:51.815253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bae5e-42d4-4112-bf44-67b78e238464 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:51.815531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bae5e-42d4-4112-bf44-67b78e238464 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:51.815806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bae5e-42d4-4112-bf44-67b78e238464 None admin] PATH_INFO: `/v3/users/5999dd8c7ba444bb898caad572d4cff1` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:51.816996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682bae5e-42d4-4112-bf44-67b78e238464 None admin] RBAC: Authorizing `identity:get_user(user_id=5999dd8c7ba444bb898caad572d4cff1)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:51.818445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682bae5e-42d4-4112-bf44-67b78e238464 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:51.820134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 596/1197] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jul 21 05:32:51 2022] GET /identity/v3/users/5999dd8c7ba444bb898caad572d4cff1 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:51.826093 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aae60e-f120-4e4c-9bd6-16db5f7675be None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:51.940438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aae60e-f120-4e4c-9bd6-16db5f7675be None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:51.953976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:51.974583 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d88955-1c95-4e94-87ea-6742d2336040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:51.975266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d88955-1c95-4e94-87ea-6742d2336040 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:51.975548 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d88955-1c95-4e94-87ea-6742d2336040 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:51.975824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d88955-1c95-4e94-87ea-6742d2336040 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:52.045017 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d88955-1c95-4e94-87ea-6742d2336040 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:52.046408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d88955-1c95-4e94-87ea-6742d2336040 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:52.145237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 602/1198] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:32:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 05:32:52.154798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bae5e-42d4-4112-bf44-67b78e238464 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:52.178218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:52.269416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682bae5e-42d4-4112-bf44-67b78e238464 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:52.305125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c086f5c-547c-41c7-a396-a52f4dca881d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:52.305926 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c086f5c-547c-41c7-a396-a52f4dca881d None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:52.306213 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c086f5c-547c-41c7-a396-a52f4dca881d None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:52.306486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c086f5c-547c-41c7-a396-a52f4dca881d None admin] PATH_INFO: `/v3/projects/e6350a5e211643c681dd82bada61f3c0` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:52.307748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c086f5c-547c-41c7-a396-a52f4dca881d None admin] RBAC: Authorizing `identity:get_project(project_id=e6350a5e211643c681dd82bada61f3c0)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:52.309200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c086f5c-547c-41c7-a396-a52f4dca881d None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:52.310928 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 597/1199] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jul 21 05:32:52 2022] GET /identity/v3/projects/e6350a5e211643c681dd82bada61f3c0 => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:32:52.320008 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d88955-1c95-4e94-87ea-6742d2336040 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:32:52.335267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:32:52.419893 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d88955-1c95-4e94-87ea-6742d2336040 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:32:52.455928 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a099e2a-103e-4f7f-9f31-5fa83fd73a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:32:52.456570 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a099e2a-103e-4f7f-9f31-5fa83fd73a1c None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:32:52.456848 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a099e2a-103e-4f7f-9f31-5fa83fd73a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:32:52.457118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a099e2a-103e-4f7f-9f31-5fa83fd73a1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:32:52.457963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a099e2a-103e-4f7f-9f31-5fa83fd73a1c None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=e6350a5e211643c681dd82bada61f3c0, user.id=5999dd8c7ba444bb898caad572d4cff1) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:32:52.458259 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a099e2a-103e-4f7f-9f31-5fa83fd73a1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:32:52.459740 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a099e2a-103e-4f7f-9f31-5fa83fd73a1c None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:32:52.465597 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 603/1200] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:32:52 2022] GET /identity/v3/role_assignments?user.id=5999dd8c7ba444bb898caad572d4cff1&scope.project.id=e6350a5e211643c681dd82bada61f3c0&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:33:02.299912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=57 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:02.462951 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=61 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:47.679060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e89dbd-95e8-4cd2-869e-eac737b85578 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:47.679060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e89dbd-95e8-4cd2-869e-eac737b85578 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:47.679060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e89dbd-95e8-4cd2-869e-eac737b85578 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:47.679862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 598/1201] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:33:47.685793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a26d20-3d96-4051-b6a8-293f26d2d074 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:47.686392 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a26d20-3d96-4051-b6a8-293f26d2d074 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:47.686392 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a26d20-3d96-4051-b6a8-293f26d2d074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:47.707667 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-e2a26d20-3d96-4051-b6a8-293f26d2d074 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:33:47.708876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-e2a26d20-3d96-4051-b6a8-293f26d2d074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:33:47.795337 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 604/1202] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:33:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:33:47.821414 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e89dbd-95e8-4cd2-869e-eac737b85578 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:47.956537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e89dbd-95e8-4cd2-869e-eac737b85578 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:47.998176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c39c91-2876-47a3-a787-738803c6cf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:47.999153 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c39c91-2876-47a3-a787-738803c6cf28 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:47.999511 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c39c91-2876-47a3-a787-738803c6cf28 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:47.999718 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c39c91-2876-47a3-a787-738803c6cf28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:48.017302 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c39c91-2876-47a3-a787-738803c6cf28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:48.019038 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c39c91-2876-47a3-a787-738803c6cf28 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:48.040250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-79c39c91-2876-47a3-a787-738803c6cf28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:33:48.040250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:33:48.040250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:33:48.040250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:33:48.040250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:33:48.040250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:33:48.040250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:33:48.040250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:33:48.040250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:33:48.040250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:33:48.040250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:33:48.040250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:33:48.040250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:33:48.040250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:33:48.040250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:33:48.041358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:33:48.041358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:33:48.041358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:33:48.041358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:33:48.041358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:33:48.041358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:33:48.041358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:33:48.041358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:33:48.041358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:33:48.041358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:33:48.041358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:33:48.041358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:33:48.041358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:33:48.041358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:33:48.041358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:33:48.041358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:33:48.041358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:33:48.042334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:33:48.042334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:33:48.042334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:33:48.042334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:33:48.042334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 599/1203] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:33:48.048523 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a26d20-3d96-4051-b6a8-293f26d2d074 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:48.168728 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a26d20-3d96-4051-b6a8-293f26d2d074 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:48.201291 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:48.208459 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45305f9-85d0-486d-85a1-abe94c3d6262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:48.209097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45305f9-85d0-486d-85a1-abe94c3d6262 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:48.209458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45305f9-85d0-486d-85a1-abe94c3d6262 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:48.209824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45305f9-85d0-486d-85a1-abe94c3d6262 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:48.210659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45305f9-85d0-486d-85a1-abe94c3d6262 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:33:48.210972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45305f9-85d0-486d-85a1-abe94c3d6262 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:48.212673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45305f9-85d0-486d-85a1-abe94c3d6262 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:48.229126 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 605/1204] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:33:48.234706 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c39c91-2876-47a3-a787-738803c6cf28 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:48.299145 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:48.363891 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c39c91-2876-47a3-a787-738803c6cf28 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:48.406247 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78156735-ff33-4112-8382-38d6d715073c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:48.407261 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78156735-ff33-4112-8382-38d6d715073c None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:48.407576 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78156735-ff33-4112-8382-38d6d715073c None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:48.407841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78156735-ff33-4112-8382-38d6d715073c None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:48.408549 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78156735-ff33-4112-8382-38d6d715073c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:48.410100 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78156735-ff33-4112-8382-38d6d715073c None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:48.434683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 600/1205] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jul 21 05:33:48 2022] POST /identity/v3/users => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:33:50.550876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4569ce-de1d-4d2c-8eb5-a437def3c9f4 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:50.551463 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4569ce-de1d-4d2c-8eb5-a437def3c9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:50.551463 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4569ce-de1d-4d2c-8eb5-a437def3c9f4 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:50.553035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 606/1206] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:33:50.558093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77610838-b919-4ff3-9b81-0db3410b6ba5 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:50.558532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77610838-b919-4ff3-9b81-0db3410b6ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:50.558736 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77610838-b919-4ff3-9b81-0db3410b6ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:50.575831 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-77610838-b919-4ff3-9b81-0db3410b6ba5 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:33:50.576997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-77610838-b919-4ff3-9b81-0db3410b6ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:33:50.653139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 601/1207] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:33:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:33:50.676601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4569ce-de1d-4d2c-8eb5-a437def3c9f4 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:50.807075 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4569ce-de1d-4d2c-8eb5-a437def3c9f4 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:50.845399 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e7a6a-8cfe-4cd8-941c-2bb96bbfba0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:50.846210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e7a6a-8cfe-4cd8-941c-2bb96bbfba0f None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:50.846585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e7a6a-8cfe-4cd8-941c-2bb96bbfba0f None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:50.846907 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e7a6a-8cfe-4cd8-941c-2bb96bbfba0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:50.858332 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e7a6a-8cfe-4cd8-941c-2bb96bbfba0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:33:50.866267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-600e7a6a-8cfe-4cd8-941c-2bb96bbfba0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:33:50.866267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:33:50.866267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:33:50.866267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:33:50.866267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:33:50.866267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:33:50.866267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:33:50.866267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:33:50.866267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:33:50.866267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:33:50.866267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:33:50.866267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:33:50.866267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:33:50.866267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:33:50.866267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:33:50.866267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:33:50.867651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:33:50.867651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:33:50.867651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:33:50.867651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:33:50.867651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:33:50.867651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:33:50.867651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:33:50.867651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:33:50.867651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:33:50.867651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:33:50.867651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:33:50.867651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:33:50.867651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:33:50.867651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:33:50.867651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:33:50.867651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:33:50.868881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:33:50.868881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:33:50.868881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:33:50.868881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:33:50.868881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:33:50.868881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:33:50.868881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:33:50.868881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:33:50.868881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:33:50.868881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:33:50.868881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:33:50.868881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:33:50.868881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 607/1208] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:33:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:33:50.873767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77610838-b919-4ff3-9b81-0db3410b6ba5 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:50.917041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:50.984326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77610838-b919-4ff3-9b81-0db3410b6ba5 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:51.026521 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf17fd-879c-47d6-ae7b-3596894328b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:51.027645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf17fd-879c-47d6-ae7b-3596894328b2 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:51.028130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf17fd-879c-47d6-ae7b-3596894328b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:51.028662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf17fd-879c-47d6-ae7b-3596894328b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:51.029519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf17fd-879c-47d6-ae7b-3596894328b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:33:51.029855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf17fd-879c-47d6-ae7b-3596894328b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:51.031510 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf17fd-879c-47d6-ae7b-3596894328b2 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:51.042491 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 602/1209] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:33:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:33:51.047975 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e7a6a-8cfe-4cd8-941c-2bb96bbfba0f None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:51.106723 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:51.161865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e7a6a-8cfe-4cd8-941c-2bb96bbfba0f None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:51.204371 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa35d9-068f-4324-a485-3e41398510b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:51.205190 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa35d9-068f-4324-a485-3e41398510b5 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:51.205475 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa35d9-068f-4324-a485-3e41398510b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:51.205773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa35d9-068f-4324-a485-3e41398510b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:51.216969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa35d9-068f-4324-a485-3e41398510b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:51.218608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa35d9-068f-4324-a485-3e41398510b5 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:51.238046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-97aa35d9-068f-4324-a485-3e41398510b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:33:51.238046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:33:51.238046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:33:51.238046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:33:51.238046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:33:51.238046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:33:51.238046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:33:51.238046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:33:51.238046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:33:51.238046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:33:51.238046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:33:51.238046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:33:51.238046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:33:51.238046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:33:51.238046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:33:51.239106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:33:51.239106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:33:51.239106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:33:51.239106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:33:51.239106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:33:51.239106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:33:51.239106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:33:51.239106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:33:51.239106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:33:51.239106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:33:51.239106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:33:51.239106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:33:51.239106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:33:51.239106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:33:51.239106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:33:51.239106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:33:51.239106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:33:51.240042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:33:51.240042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:33:51.240042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:33:51.240042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:33:51.240042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 608/1210] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:33:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:33:51.245531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf17fd-879c-47d6-ae7b-3596894328b2 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:51.363925 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:51.368054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf17fd-879c-47d6-ae7b-3596894328b2 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:51.409446 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267035c-2040-4551-aa3b-609789cca2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:51.409446 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267035c-2040-4551-aa3b-609789cca2ad None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:51.409446 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267035c-2040-4551-aa3b-609789cca2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:51.409446 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267035c-2040-4551-aa3b-609789cca2ad None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:51.409901 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267035c-2040-4551-aa3b-609789cca2ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:33:51.409901 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267035c-2040-4551-aa3b-609789cca2ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:51.411163 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267035c-2040-4551-aa3b-609789cca2ad None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:51.426103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 603/1211] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:33:51.432662 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa35d9-068f-4324-a485-3e41398510b5 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:51.557013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa35d9-068f-4324-a485-3e41398510b5 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:51.558843 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:51.596458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58a934-e113-4c9e-bce1-d329538d15eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:51.597346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58a934-e113-4c9e-bce1-d329538d15eb None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:51.597645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58a934-e113-4c9e-bce1-d329538d15eb None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:51.597922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58a934-e113-4c9e-bce1-d329538d15eb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:51.606208 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58a934-e113-4c9e-bce1-d329538d15eb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:51.607836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58a934-e113-4c9e-bce1-d329538d15eb None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:51.624056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-9b58a934-e113-4c9e-bce1-d329538d15eb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 05:33:51.624056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:33:51.624056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:33:51.624056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:33:51.624056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:33:51.624056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:33:51.624056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:33:51.624056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:33:51.624056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:33:51.624056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:33:51.624056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:33:51.624056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:33:51.624056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:33:51.624056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:33:51.624056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:33:51.624056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:33:51.625409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:33:51.626387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:33:51.626387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:33:51.626387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:33:51.626387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:33:51.626387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:33:51.626387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:33:51.626387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:33:51.626387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:33:51.626387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 05:33:51.626387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:33:51.630428 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 609/1212] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:33:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:33:51.631804 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267035c-2040-4551-aa3b-609789cca2ad None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:51.751392 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267035c-2040-4551-aa3b-609789cca2ad None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:51.787553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534e249-8b40-4143-a9f5-7f5ea33e9ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:51.788251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534e249-8b40-4143-a9f5-7f5ea33e9ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:51.788545 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534e249-8b40-4143-a9f5-7f5ea33e9ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:51.788785 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534e249-8b40-4143-a9f5-7f5ea33e9ff5 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:51.789711 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534e249-8b40-4143-a9f5-7f5ea33e9ff5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:33:51.790047 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534e249-8b40-4143-a9f5-7f5ea33e9ff5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:51.791606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534e249-8b40-4143-a9f5-7f5ea33e9ff5 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:51.810916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 604/1213] 10.222.0.44 () {56 vars in 1143 bytes} [Thu Jul 21 05:33:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:33:51.817346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58a934-e113-4c9e-bce1-d329538d15eb None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:51.927143 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58a934-e113-4c9e-bce1-d329538d15eb None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:51.970776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628f81e-f7ca-4273-bdc5-867c55437ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:51.971561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628f81e-f7ca-4273-bdc5-867c55437ced None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:51.971927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628f81e-f7ca-4273-bdc5-867c55437ced None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:51.972228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4628f81e-f7ca-4273-bdc5-867c55437ced None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:52.008480 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:52.063383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628f81e-f7ca-4273-bdc5-867c55437ced None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:52.065013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4628f81e-f7ca-4273-bdc5-867c55437ced None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:52.189641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 610/1214] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:33:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 05:33:52.192057 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534e249-8b40-4143-a9f5-7f5ea33e9ff5 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:52.213334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:52.322730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534e249-8b40-4143-a9f5-7f5ea33e9ff5 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:52.363605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab3056-faa6-49b2-867d-6e6774f18493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:52.364414 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab3056-faa6-49b2-867d-6e6774f18493 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:52.364725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab3056-faa6-49b2-867d-6e6774f18493 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:52.365170 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab3056-faa6-49b2-867d-6e6774f18493 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:52.376474 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab3056-faa6-49b2-867d-6e6774f18493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:52.378059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab3056-faa6-49b2-867d-6e6774f18493 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:52.400009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-2dab3056-faa6-49b2-867d-6e6774f18493 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:33:52.400009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:33:52.400009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:33:52.400009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:33:52.400009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:33:52.400009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:33:52.400009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:33:52.400009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:33:52.400009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:33:52.400009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:33:52.400009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:33:52.400009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:33:52.400009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:33:52.400009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:33:52.400009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:33:52.401195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:33:52.401195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:33:52.401195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:33:52.401195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:33:52.401195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:33:52.401195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:33:52.401195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:33:52.401195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:33:52.401195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:33:52.401195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:33:52.401195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:33:52.401195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:33:52.401195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:33:52.401195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:33:52.401195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:33:52.401195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:33:52.401195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:33:52.407322 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:33:52.407322 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:33:52.407322 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:33:52.407322 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:33:52.407322 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 605/1215] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:33:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:33:52.408472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628f81e-f7ca-4273-bdc5-867c55437ced None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:52.481118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:52.525633 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4628f81e-f7ca-4273-bdc5-867c55437ced None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:52.564615 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68b20e-65e4-4674-9f74-4ba1cca3b632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:52.565395 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68b20e-65e4-4674-9f74-4ba1cca3b632 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:52.565560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68b20e-65e4-4674-9f74-4ba1cca3b632 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:52.565925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68b20e-65e4-4674-9f74-4ba1cca3b632 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:52.566646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68b20e-65e4-4674-9f74-4ba1cca3b632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:33:52.566964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68b20e-65e4-4674-9f74-4ba1cca3b632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:52.568403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68b20e-65e4-4674-9f74-4ba1cca3b632 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:52.581081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 611/1216] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:33:52.586924 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab3056-faa6-49b2-867d-6e6774f18493 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:52.672334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:52.713395 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab3056-faa6-49b2-867d-6e6774f18493 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:52.754861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab26c07-25c2-4236-b18d-8620a202bd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:52.755604 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab26c07-25c2-4236-b18d-8620a202bd9f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:52.755884 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab26c07-25c2-4236-b18d-8620a202bd9f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:52.756154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab26c07-25c2-4236-b18d-8620a202bd9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:52.763588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab26c07-25c2-4236-b18d-8620a202bd9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:52.765152 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab26c07-25c2-4236-b18d-8620a202bd9f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:52.779830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-dab26c07-25c2-4236-b18d-8620a202bd9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:33:52.779830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:33:52.779830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:33:52.779830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:33:52.779830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:33:52.779830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:33:52.779830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:33:52.779830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:33:52.779830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:33:52.779830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:33:52.779830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:33:52.779830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:33:52.779830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:33:52.779830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:33:52.779830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:33:52.781060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:33:52.781060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:33:52.781060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:33:52.781060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:33:52.781060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:33:52.781060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:33:52.781060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:33:52.781060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:33:52.781060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:33:52.781060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:33:52.781060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:33:52.781060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:33:52.781060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:33:52.781060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:33:52.781060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:33:52.781060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:33:52.781060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:33:52.782009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:33:52.782009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:33:52.782009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:33:52.782009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:33:52.782009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:33:52.782009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:33:52.782009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:33:52.782009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:33:52.782009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 606/1217] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:33:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:33:52.787939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68b20e-65e4-4674-9f74-4ba1cca3b632 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:52.899923 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68b20e-65e4-4674-9f74-4ba1cca3b632 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:52.925198 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:52.938403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5b452-eadd-4aec-b5ec-56a5c33ff400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:52.939226 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5b452-eadd-4aec-b5ec-56a5c33ff400 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:52.939607 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5b452-eadd-4aec-b5ec-56a5c33ff400 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:52.939885 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5b452-eadd-4aec-b5ec-56a5c33ff400 None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:52.940734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5b452-eadd-4aec-b5ec-56a5c33ff400 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:33:52.941048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5b452-eadd-4aec-b5ec-56a5c33ff400 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:52.942482 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5b452-eadd-4aec-b5ec-56a5c33ff400 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:52.955861 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 612/1218] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:33:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:33:52.962516 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab26c07-25c2-4236-b18d-8620a202bd9f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:53.069836 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab26c07-25c2-4236-b18d-8620a202bd9f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:53.101676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:53.108288 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f1e66-b779-4e1b-a8fe-60d93f1c7c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:53.108957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f1e66-b779-4e1b-a8fe-60d93f1c7c0d None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:53.109238 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f1e66-b779-4e1b-a8fe-60d93f1c7c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:53.109509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f1e66-b779-4e1b-a8fe-60d93f1c7c0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:53.110426 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f1e66-b779-4e1b-a8fe-60d93f1c7c0d None admin] RBAC: Adding query filter params (role.id=bb94781ab8054924a5cb21b561b12bbb, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=03d868598e39448881b43d299253d821) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:33:53.110743 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f1e66-b779-4e1b-a8fe-60d93f1c7c0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:53.112331 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f1e66-b779-4e1b-a8fe-60d93f1c7c0d None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:53.119092 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 607/1219] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:33:52 2022] GET /identity/v3/role_assignments?user.id=03d868598e39448881b43d299253d821&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=bb94781ab8054924a5cb21b561b12bbb => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:33:55.142365 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2885e-d2a8-4bec-b25f-6f70cfbe32fb None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:55.142927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2885e-d2a8-4bec-b25f-6f70cfbe32fb None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:55.142927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2885e-d2a8-4bec-b25f-6f70cfbe32fb None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:55.144356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 613/1220] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:33:55.150597 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff246c-8095-4791-9419-8cc11282810a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:55.151141 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff246c-8095-4791-9419-8cc11282810a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:55.151232 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff246c-8095-4791-9419-8cc11282810a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:55.171628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-1fff246c-8095-4791-9419-8cc11282810a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:33:55.173046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1fff246c-8095-4791-9419-8cc11282810a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:33:55.246275 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 608/1221] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:33:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:33:55.270477 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2885e-d2a8-4bec-b25f-6f70cfbe32fb None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:55.382406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2885e-d2a8-4bec-b25f-6f70cfbe32fb None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:55.420091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a51710-8ccb-4393-9372-4d81584d65a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:55.420972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a51710-8ccb-4393-9372-4d81584d65a1 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:55.421299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a51710-8ccb-4393-9372-4d81584d65a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:55.421639 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a51710-8ccb-4393-9372-4d81584d65a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:55.435732 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a51710-8ccb-4393-9372-4d81584d65a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:33:55.445313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-98a51710-8ccb-4393-9372-4d81584d65a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:33:55.445313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:33:55.445313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:33:55.445313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:33:55.445313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:33:55.445313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:33:55.445313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:33:55.445313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:33:55.445313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:33:55.445313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:33:55.445313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:33:55.445313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:33:55.445313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:33:55.445313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:33:55.445313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:33:55.445313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:33:55.446803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:33:55.446803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:33:55.446803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:33:55.446803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:33:55.446803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:33:55.446803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:33:55.446803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:33:55.446803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:33:55.446803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:33:55.446803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:33:55.446803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:33:55.446803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:33:55.446803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:33:55.446803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:33:55.446803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:33:55.446803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:33:55.447739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:33:55.447739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:33:55.447739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:33:55.447739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:33:55.447739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:33:55.447739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:33:55.447739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:33:55.447739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:33:55.447739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:33:55.447739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:33:55.447739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:33:55.447739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:33:55.447739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 614/1222] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:33:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:33:55.453328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff246c-8095-4791-9419-8cc11282810a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:55.565619 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff246c-8095-4791-9419-8cc11282810a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:55.592196 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:55.605596 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efb5cc-2b99-459d-a9b0-a77a3ceac3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:55.606458 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efb5cc-2b99-459d-a9b0-a77a3ceac3b5 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:55.607000 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efb5cc-2b99-459d-a9b0-a77a3ceac3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:55.607378 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efb5cc-2b99-459d-a9b0-a77a3ceac3b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:55.608218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efb5cc-2b99-459d-a9b0-a77a3ceac3b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:33:55.608528 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efb5cc-2b99-459d-a9b0-a77a3ceac3b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:55.610200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efb5cc-2b99-459d-a9b0-a77a3ceac3b5 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:55.626485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 609/1223] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:33:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:33:55.633456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a51710-8ccb-4393-9372-4d81584d65a1 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:55.687292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:55.749091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a51710-8ccb-4393-9372-4d81584d65a1 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:55.788994 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1b558-e3e3-4125-9169-0f8e42a33363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:55.789851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1b558-e3e3-4125-9169-0f8e42a33363 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:55.790113 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1b558-e3e3-4125-9169-0f8e42a33363 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:55.790436 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1b558-e3e3-4125-9169-0f8e42a33363 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:55.804362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1b558-e3e3-4125-9169-0f8e42a33363 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:55.805920 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1b558-e3e3-4125-9169-0f8e42a33363 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:55.825215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-97a1b558-e3e3-4125-9169-0f8e42a33363 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:33:55.825215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:33:55.825215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:33:55.825215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:33:55.825215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:33:55.825215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:33:55.825215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:33:55.825215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:33:55.825215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:33:55.825215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:33:55.825215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:33:55.825215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:33:55.825215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:33:55.825215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:33:55.825215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:33:55.826272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:33:55.826272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:33:55.826272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:33:55.826272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:33:55.826272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:33:55.826272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:33:55.826272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:33:55.826272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:33:55.826272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:33:55.826272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:33:55.826272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:33:55.826272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:33:55.826272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:33:55.826272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:33:55.826272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:33:55.826272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:33:55.826272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:33:55.827219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:33:55.827219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:33:55.827219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:33:55.827219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:33:55.827219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 615/1224] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:33:55.833380 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efb5cc-2b99-459d-a9b0-a77a3ceac3b5 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:55.943603 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efb5cc-2b99-459d-a9b0-a77a3ceac3b5 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:55.981267 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a7494-10a3-40b4-9756-457d4bc5ba59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:55.981939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a7494-10a3-40b4-9756-457d4bc5ba59 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:55.982221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a7494-10a3-40b4-9756-457d4bc5ba59 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:55.982494 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a7494-10a3-40b4-9756-457d4bc5ba59 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:55.983256 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a7494-10a3-40b4-9756-457d4bc5ba59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:33:55.983566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a7494-10a3-40b4-9756-457d4bc5ba59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:55.984948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a7494-10a3-40b4-9756-457d4bc5ba59 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:55.998158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 610/1225] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:33:56.004998 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1b558-e3e3-4125-9169-0f8e42a33363 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:56.109355 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:56.119742 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1b558-e3e3-4125-9169-0f8e42a33363 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:56.156247 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72adf103-6c48-4fe9-a694-c596e135c3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:56.157098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adf103-6c48-4fe9-a694-c596e135c3ae None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:56.157369 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adf103-6c48-4fe9-a694-c596e135c3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:56.157673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adf103-6c48-4fe9-a694-c596e135c3ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:56.166931 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72adf103-6c48-4fe9-a694-c596e135c3ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:56.168581 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72adf103-6c48-4fe9-a694-c596e135c3ae None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:56.184585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-72adf103-6c48-4fe9-a694-c596e135c3ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 05:33:56.184585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:33:56.184585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:33:56.184585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:33:56.184585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:33:56.184585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:33:56.184585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:33:56.184585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:33:56.184585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:33:56.184585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:33:56.184585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:33:56.184585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:33:56.184585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:33:56.184585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:33:56.184585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:33:56.184585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:33:56.185586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:33:56.186925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:33:56.186925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:33:56.186925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:33:56.186925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:33:56.186925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:33:56.186925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:33:56.186925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:33:56.186925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:33:56.186925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 05:33:56.186925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:33:56.186925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 616/1226] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:33:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:33:56.192560 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a7494-10a3-40b4-9756-457d4bc5ba59 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:56.240846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:56.293502 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a7494-10a3-40b4-9756-457d4bc5ba59 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:56.335755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06eb61-7402-4dde-81cd-39af7e6851a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:56.336448 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06eb61-7402-4dde-81cd-39af7e6851a2 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:56.336733 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06eb61-7402-4dde-81cd-39af7e6851a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:56.337017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06eb61-7402-4dde-81cd-39af7e6851a2 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:56.337899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06eb61-7402-4dde-81cd-39af7e6851a2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:33:56.338202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06eb61-7402-4dde-81cd-39af7e6851a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:56.339837 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06eb61-7402-4dde-81cd-39af7e6851a2 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:56.358420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 611/1227] 10.222.0.44 () {56 vars in 1143 bytes} [Thu Jul 21 05:33:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:33:56.365137 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72adf103-6c48-4fe9-a694-c596e135c3ae None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:56.476398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72adf103-6c48-4fe9-a694-c596e135c3ae None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:56.510607 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39365f84-6b07-427d-a6a1-5a87169aec50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:56.511373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39365f84-6b07-427d-a6a1-5a87169aec50 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:56.511659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39365f84-6b07-427d-a6a1-5a87169aec50 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:56.511938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39365f84-6b07-427d-a6a1-5a87169aec50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:56.522824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39365f84-6b07-427d-a6a1-5a87169aec50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:56.524344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39365f84-6b07-427d-a6a1-5a87169aec50 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:56.542106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-39365f84-6b07-427d-a6a1-5a87169aec50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:33:56.542106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:33:56.542106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:33:56.542106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:33:56.542106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:33:56.542106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:33:56.542106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:33:56.542106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:33:56.542106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:33:56.542106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:33:56.542106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:33:56.542106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:33:56.542106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:33:56.542106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:33:56.542106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:33:56.543232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:33:56.543232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:33:56.543232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:33:56.543232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:33:56.543232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:33:56.543232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:33:56.543232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:33:56.543232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:33:56.543232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:33:56.543232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:33:56.543232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:33:56.543232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:33:56.543232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:33:56.543232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:33:56.543232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:33:56.543232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:33:56.543232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:33:56.544451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:33:56.544451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:33:56.544451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:33:56.544451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:33:56.544451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 617/1228] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:33:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:33:56.549365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06eb61-7402-4dde-81cd-39af7e6851a2 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:56.655289 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06eb61-7402-4dde-81cd-39af7e6851a2 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:56.658022 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:56.695693 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576be278-34f8-4802-b21c-accaf1c8ed75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:56.697380 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576be278-34f8-4802-b21c-accaf1c8ed75 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:56.697769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576be278-34f8-4802-b21c-accaf1c8ed75 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:56.698003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576be278-34f8-4802-b21c-accaf1c8ed75 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:56.698957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576be278-34f8-4802-b21c-accaf1c8ed75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:33:56.699296 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576be278-34f8-4802-b21c-accaf1c8ed75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:56.700744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576be278-34f8-4802-b21c-accaf1c8ed75 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:56.713134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 612/1229] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:33:56.719378 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39365f84-6b07-427d-a6a1-5a87169aec50 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:56.729173 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:56.829726 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39365f84-6b07-427d-a6a1-5a87169aec50 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:56.865620 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805c76e-90cd-480d-821d-347c544087b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:56.866444 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805c76e-90cd-480d-821d-347c544087b5 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:56.866733 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805c76e-90cd-480d-821d-347c544087b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:56.867040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805c76e-90cd-480d-821d-347c544087b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:56.874674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805c76e-90cd-480d-821d-347c544087b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:56.876400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805c76e-90cd-480d-821d-347c544087b5 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:56.892976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-9805c76e-90cd-480d-821d-347c544087b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:33:56.892976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:33:56.892976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:33:56.892976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:33:56.892976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:33:56.892976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:33:56.892976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:33:56.892976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:33:56.892976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:33:56.892976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:33:56.892976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:33:56.892976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:33:56.892976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:33:56.892976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:33:56.892976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:33:56.894221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:33:56.894221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:33:56.894221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:33:56.894221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:33:56.894221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:33:56.894221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:33:56.894221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:33:56.894221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:33:56.894221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:33:56.894221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:33:56.894221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:33:56.894221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:33:56.894221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:33:56.894221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:33:56.894221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:33:56.894221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:33:56.894221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:33:56.895288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:33:56.895288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:33:56.895288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:33:56.895288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:33:56.895288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:33:56.895288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:33:56.895288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:33:56.895288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:33:56.895288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 618/1230] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:33:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:33:56.900735 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576be278-34f8-4802-b21c-accaf1c8ed75 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:57.010771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576be278-34f8-4802-b21c-accaf1c8ed75 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:57.050153 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d731e-ffb5-4dc9-9ef9-f981977b40f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:57.050911 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d731e-ffb5-4dc9-9ef9-f981977b40f2 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:57.051321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d731e-ffb5-4dc9-9ef9-f981977b40f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:57.051585 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d731e-ffb5-4dc9-9ef9-f981977b40f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:57.052460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d731e-ffb5-4dc9-9ef9-f981977b40f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:33:57.052828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d731e-ffb5-4dc9-9ef9-f981977b40f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:57.054301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d731e-ffb5-4dc9-9ef9-f981977b40f2 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:57.068000 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 613/1231] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:33:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:33:57.074396 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805c76e-90cd-480d-821d-347c544087b5 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:57.148253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:57.186876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805c76e-90cd-480d-821d-347c544087b5 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:57.222561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95246ee5-73b7-48df-a823-42051464f0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:57.223449 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95246ee5-73b7-48df-a823-42051464f0b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:57.223712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95246ee5-73b7-48df-a823-42051464f0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:57.224134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95246ee5-73b7-48df-a823-42051464f0b1 None admin] PATH_INFO: `/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/03d868598e39448881b43d299253d821/roles/bb94781ab8054924a5cb21b561b12bbb` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:57.239782 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95246ee5-73b7-48df-a823-42051464f0b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b6f27b8d9b44d191210a24a9e04041, user_id=03d868598e39448881b43d299253d821, role_id=bb94781ab8054924a5cb21b561b12bbb)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:57.241418 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95246ee5-73b7-48df-a823-42051464f0b1 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:57.253686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 619/1232] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:33:57 2022] PUT /identity/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/03d868598e39448881b43d299253d821/roles/bb94781ab8054924a5cb21b561b12bbb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:33:59.305253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ea5b4-92a3-4863-a7d8-75f40421c0df None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:59.305807 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ea5b4-92a3-4863-a7d8-75f40421c0df None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:59.305807 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ea5b4-92a3-4863-a7d8-75f40421c0df None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:59.307216 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 614/1233] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:33:59.313757 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffea89-e4d7-4398-8d0b-c2ce03e596ef None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:59.314319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffea89-e4d7-4398-8d0b-c2ce03e596ef None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:59.314319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffea89-e4d7-4398-8d0b-c2ce03e596ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:59.330846 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-ceffea89-e4d7-4398-8d0b-c2ce03e596ef None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:33:59.332086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ceffea89-e4d7-4398-8d0b-c2ce03e596ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:33:59.408639 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 620/1234] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:33:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:33:59.434695 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ea5b4-92a3-4863-a7d8-75f40421c0df None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:59.464627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:59.552030 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ea5b4-92a3-4863-a7d8-75f40421c0df None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:59.586085 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2bb8f-267a-422e-ba50-1610f22542b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:59.586886 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2bb8f-267a-422e-ba50-1610f22542b4 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:59.587128 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2bb8f-267a-422e-ba50-1610f22542b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:59.587403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2bb8f-267a-422e-ba50-1610f22542b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:59.598444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2bb8f-267a-422e-ba50-1610f22542b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:33:59.606722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-a4a2bb8f-267a-422e-ba50-1610f22542b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:33:59.606722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:33:59.606722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:33:59.606722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:33:59.606722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:33:59.606722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:33:59.606722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:33:59.606722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:33:59.606722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:33:59.606722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:33:59.606722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:33:59.606722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:33:59.606722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:33:59.606722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:33:59.606722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:33:59.606722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:33:59.607945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:33:59.607945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:33:59.607945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:33:59.607945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:33:59.607945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:33:59.607945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:33:59.607945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:33:59.607945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:33:59.607945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:33:59.607945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:33:59.607945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:33:59.607945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:33:59.607945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:33:59.607945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:33:59.607945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:33:59.607945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:33:59.608808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:33:59.608808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:33:59.608808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:33:59.608808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:33:59.608808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:33:59.608808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:33:59.608808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:33:59.608808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:33:59.608808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:33:59.608808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:33:59.608808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 21 05:33:59.608808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:33:59.608808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 615/1235] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jul 21 05:33:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:33:59.614638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffea89-e4d7-4398-8d0b-c2ce03e596ef None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:59.665131 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:59.720308 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffea89-e4d7-4398-8d0b-c2ce03e596ef None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:59.756899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5706a9-676c-47a1-b161-c3c2dce90431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:59.757581 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5706a9-676c-47a1-b161-c3c2dce90431 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:59.757865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5706a9-676c-47a1-b161-c3c2dce90431 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:59.758145 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5706a9-676c-47a1-b161-c3c2dce90431 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:59.758887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5706a9-676c-47a1-b161-c3c2dce90431 None admin] RBAC: Adding query filter params (name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:33:59.759229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5706a9-676c-47a1-b161-c3c2dce90431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:59.760618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5706a9-676c-47a1-b161-c3c2dce90431 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:59.769034 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 621/1236] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jul 21 05:33:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:33:59.775355 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2bb8f-267a-422e-ba50-1610f22542b4 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:33:59.874436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:33:59.892860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2bb8f-267a-422e-ba50-1610f22542b4 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:33:59.932367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bc911-1c33-44d2-bf00-e13e8dc214fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:33:59.933093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bc911-1c33-44d2-bf00-e13e8dc214fd None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:33:59.933377 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bc911-1c33-44d2-bf00-e13e8dc214fd None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:33:59.933668 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bc911-1c33-44d2-bf00-e13e8dc214fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:33:59.945334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328bc911-1c33-44d2-bf00-e13e8dc214fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:33:59.946864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328bc911-1c33-44d2-bf00-e13e8dc214fd None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:33:59.964642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-328bc911-1c33-44d2-bf00-e13e8dc214fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:33:59.964642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:33:59.964642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:33:59.964642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:33:59.964642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:33:59.964642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:33:59.964642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:33:59.964642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:33:59.964642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:33:59.964642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:33:59.964642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:33:59.964642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:33:59.964642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:33:59.964642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:33:59.964642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:33:59.965745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:33:59.965745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:33:59.965745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:33:59.965745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:33:59.965745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:33:59.965745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:33:59.965745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:33:59.965745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:33:59.965745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:33:59.965745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:33:59.965745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:33:59.965745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:33:59.965745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:33:59.965745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:33:59.965745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:33:59.965745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:33:59.965745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:33:59.966816 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:33:59.966816 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:33:59.966816 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:33:59.966816 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:33:59.966816 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 616/1237] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:33:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:33:59.972121 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5706a9-676c-47a1-b161-c3c2dce90431 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:00.077389 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5706a9-676c-47a1-b161-c3c2dce90431 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:00.087932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:00.115691 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e971f-5b9e-4497-ad4a-a9015b9d34f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:00.116360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e971f-5b9e-4497-ad4a-a9015b9d34f1 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:00.116641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e971f-5b9e-4497-ad4a-a9015b9d34f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:00.116912 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e971f-5b9e-4497-ad4a-a9015b9d34f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:00.117628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e971f-5b9e-4497-ad4a-a9015b9d34f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:00.117930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e971f-5b9e-4497-ad4a-a9015b9d34f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:00.119337 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27e971f-5b9e-4497-ad4a-a9015b9d34f1 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:00.131673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 622/1238] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:00.137460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bc911-1c33-44d2-bf00-e13e8dc214fd None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:00.245123 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bc911-1c33-44d2-bf00-e13e8dc214fd None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:00.286686 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721dfae-9636-4503-9e02-9804e1acf991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:00.287518 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721dfae-9636-4503-9e02-9804e1acf991 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:00.287799 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721dfae-9636-4503-9e02-9804e1acf991 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:00.288100 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721dfae-9636-4503-9e02-9804e1acf991 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:00.290934 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:00.296564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6721dfae-9636-4503-9e02-9804e1acf991 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:00.298407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6721dfae-9636-4503-9e02-9804e1acf991 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:00.312776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-6721dfae-9636-4503-9e02-9804e1acf991 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 05:34:00.312776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:00.312776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:00.312776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:00.312776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:00.312776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:00.312776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:00.312776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:00.312776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:00.312776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:00.312776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:00.312776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:00.312776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:34:00.312776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:34:00.312776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:34:00.312776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:00.314531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:00.315702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:00.315702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:00.315702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:34:00.315702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:34:00.315702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:34:00.315702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:34:00.315702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:34:00.315702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:34:00.315702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 05:34:00.315702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:34:00.315702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 617/1239] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:34:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:34:00.320204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e971f-5b9e-4497-ad4a-a9015b9d34f1 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:00.431594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e971f-5b9e-4497-ad4a-a9015b9d34f1 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:00.469907 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bffa16-ca27-49fe-8dfd-968506cb4ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:00.470633 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bffa16-ca27-49fe-8dfd-968506cb4ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:00.470902 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bffa16-ca27-49fe-8dfd-968506cb4ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:00.471279 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bffa16-ca27-49fe-8dfd-968506cb4ff3 None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:00.472104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bffa16-ca27-49fe-8dfd-968506cb4ff3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:00.472411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bffa16-ca27-49fe-8dfd-968506cb4ff3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:00.473860 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bffa16-ca27-49fe-8dfd-968506cb4ff3 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:00.492707 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 623/1240] 10.222.0.44 () {56 vars in 1143 bytes} [Thu Jul 21 05:34:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:34:00.498530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721dfae-9636-4503-9e02-9804e1acf991 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:00.606257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721dfae-9636-4503-9e02-9804e1acf991 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:00.641347 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dc530-d429-4f0b-b40c-54d660b70254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:00.642013 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dc530-d429-4f0b-b40c-54d660b70254 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:00.642294 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dc530-d429-4f0b-b40c-54d660b70254 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:00.642639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dc530-d429-4f0b-b40c-54d660b70254 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:00.721509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:00.730085 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81dc530-d429-4f0b-b40c-54d660b70254 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:00.732129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81dc530-d429-4f0b-b40c-54d660b70254 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:00.841553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 618/1241] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:34:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 21 05:34:00.847641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bffa16-ca27-49fe-8dfd-968506cb4ff3 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:00.877389 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:00.950598 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bffa16-ca27-49fe-8dfd-968506cb4ff3 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:00.988217 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ceb35-2d78-4c53-be64-73b0b66693d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:00.988919 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ceb35-2d78-4c53-be64-73b0b66693d0 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:00.989221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ceb35-2d78-4c53-be64-73b0b66693d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:00.989508 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ceb35-2d78-4c53-be64-73b0b66693d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:01.000262 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ceb35-2d78-4c53-be64-73b0b66693d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:01.001751 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ceb35-2d78-4c53-be64-73b0b66693d0 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:01.019682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-403ceb35-2d78-4c53-be64-73b0b66693d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:34:01.019682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:01.019682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:01.019682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:01.019682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:01.019682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:01.019682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:01.019682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:01.019682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:01.019682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:01.019682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:01.019682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:01.019682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:34:01.019682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:34:01.019682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:34:01.020830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:34:01.020830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:01.020830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:01.020830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:01.020830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:01.020830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:01.020830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:01.020830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:01.020830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:01.020830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:01.020830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:01.020830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:01.020830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:01.020830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:01.020830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:01.020830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:01.020830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:01.022070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:34:01.022070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:34:01.022070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:34:01.022070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:34:01.022070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 624/1242] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:34:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:01.027063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dc530-d429-4f0b-b40c-54d660b70254 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:01.128363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:01.135248 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dc530-d429-4f0b-b40c-54d660b70254 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:01.172431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5bde6-20ad-4cf7-aea7-8242c29439bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:01.173906 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5bde6-20ad-4cf7-aea7-8242c29439bc None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:01.174437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5bde6-20ad-4cf7-aea7-8242c29439bc None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:01.174931 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5bde6-20ad-4cf7-aea7-8242c29439bc None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:01.176108 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5bde6-20ad-4cf7-aea7-8242c29439bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:01.176525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5bde6-20ad-4cf7-aea7-8242c29439bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:01.178773 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5bde6-20ad-4cf7-aea7-8242c29439bc None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:01.193643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 619/1243] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:01.199546 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ceb35-2d78-4c53-be64-73b0b66693d0 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:01.292115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:01.317803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ceb35-2d78-4c53-be64-73b0b66693d0 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:01.358840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94056e3d-0f67-4638-ac21-78f095be6108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:01.359677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94056e3d-0f67-4638-ac21-78f095be6108 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:01.359956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94056e3d-0f67-4638-ac21-78f095be6108 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:01.360219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94056e3d-0f67-4638-ac21-78f095be6108 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:01.368456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94056e3d-0f67-4638-ac21-78f095be6108 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:01.370115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94056e3d-0f67-4638-ac21-78f095be6108 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:01.386005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-94056e3d-0f67-4638-ac21-78f095be6108 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:34:01.386005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:01.386005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:01.386005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:01.386005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:01.386005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:01.386005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:01.386005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:01.386005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:01.386005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:01.386005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:01.386005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:01.386005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:34:01.386005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:34:01.386005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:34:01.387111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:34:01.387111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:01.387111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:01.387111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:01.387111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:01.387111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:01.387111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:01.387111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:01.387111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:01.387111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:01.387111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:01.387111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:01.387111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:01.387111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:01.387111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:01.387111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:01.387111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:01.388200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:34:01.388200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:34:01.388200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:34:01.388200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:34:01.388200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:34:01.388200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:34:01.388200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:34:01.388200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:34:01.388200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 625/1244] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:34:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:01.394038 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5bde6-20ad-4cf7-aea7-8242c29439bc None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:01.496243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5bde6-20ad-4cf7-aea7-8242c29439bc None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:01.530589 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963916f-3bc3-4eed-9775-2a850d69d54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:01.531306 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963916f-3bc3-4eed-9775-2a850d69d54b None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:01.531590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963916f-3bc3-4eed-9775-2a850d69d54b None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:01.531863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963916f-3bc3-4eed-9775-2a850d69d54b None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:01.532654 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963916f-3bc3-4eed-9775-2a850d69d54b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:01.532964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963916f-3bc3-4eed-9775-2a850d69d54b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:01.534631 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963916f-3bc3-4eed-9775-2a850d69d54b None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:01.549379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 620/1245] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:34:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:01.555690 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94056e3d-0f67-4638-ac21-78f095be6108 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:01.666218 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94056e3d-0f67-4638-ac21-78f095be6108 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:01.704702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b681c62-94d1-45f3-aee5-f0b0995df54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:01.705741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b681c62-94d1-45f3-aee5-f0b0995df54e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:01.706179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b681c62-94d1-45f3-aee5-f0b0995df54e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:01.706516 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b681c62-94d1-45f3-aee5-f0b0995df54e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:01.707469 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b681c62-94d1-45f3-aee5-f0b0995df54e None admin] RBAC: Adding query filter params (role.id=bb94781ab8054924a5cb21b561b12bbb, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=03d868598e39448881b43d299253d821) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:01.707776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b681c62-94d1-45f3-aee5-f0b0995df54e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:01.709726 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b681c62-94d1-45f3-aee5-f0b0995df54e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:01.717552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 626/1246] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:34:01 2022] GET /identity/v3/role_assignments?user.id=03d868598e39448881b43d299253d821&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=bb94781ab8054924a5cb21b561b12bbb => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:03.990846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c04288-b52f-44e5-9dde-b0463e0f717e None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:03.991390 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c04288-b52f-44e5-9dde-b0463e0f717e None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:03.991390 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c04288-b52f-44e5-9dde-b0463e0f717e None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:03.992741 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 621/1247] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:34:03.998354 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a60ae-6427-490a-9b0b-0e744974e4e2 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:03.998867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a60ae-6427-490a-9b0b-0e744974e4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:03.998994 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a60ae-6427-490a-9b0b-0e744974e4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:04.004733 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:04.018371 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-555a60ae-6427-490a-9b0b-0e744974e4e2 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:34:04.019760 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-555a60ae-6427-490a-9b0b-0e744974e4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:34:04.092957 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 627/1248] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:34:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:34:04.119214 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c04288-b52f-44e5-9dde-b0463e0f717e None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:04.140036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:04.231375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c04288-b52f-44e5-9dde-b0463e0f717e None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:04.267318 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727268b-49da-4b58-8fc3-60e32fb2a032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:04.268129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727268b-49da-4b58-8fc3-60e32fb2a032 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:04.268411 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727268b-49da-4b58-8fc3-60e32fb2a032 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:04.268682 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727268b-49da-4b58-8fc3-60e32fb2a032 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:04.279481 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f727268b-49da-4b58-8fc3-60e32fb2a032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:34:04.287476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-f727268b-49da-4b58-8fc3-60e32fb2a032 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:34:04.287476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:04.287476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:04.287476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:04.287476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:04.287476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:04.287476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:04.287476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:04.287476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:04.287476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:04.287476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:04.287476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:04.287476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:34:04.287476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:34:04.287476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:34:04.287476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:34:04.288627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:34:04.288627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:34:04.288627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:34:04.288627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:34:04.288627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:04.288627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:04.288627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:04.288627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:04.288627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:04.288627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:04.288627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:04.288627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:04.288627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:04.288627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:04.288627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:04.288627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:04.289732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:04.289732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:04.289732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:04.289732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:04.289732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:34:04.289732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:34:04.289732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:34:04.289732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:34:04.289732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:34:04.289732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:34:04.289732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:34:04.289732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:34:04.289732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 622/1249] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:34:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:04.295913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a60ae-6427-490a-9b0b-0e744974e4e2 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:04.405813 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555a60ae-6427-490a-9b0b-0e744974e4e2 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:04.440989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:04.447364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60fe91-3b35-4079-95fa-613d9759c843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:04.448088 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60fe91-3b35-4079-95fa-613d9759c843 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:04.448453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60fe91-3b35-4079-95fa-613d9759c843 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:04.448628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60fe91-3b35-4079-95fa-613d9759c843 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:04.449380 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca60fe91-3b35-4079-95fa-613d9759c843 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:04.449725 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca60fe91-3b35-4079-95fa-613d9759c843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:04.451146 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca60fe91-3b35-4079-95fa-613d9759c843 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:04.461814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 628/1250] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:34:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:04.469208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727268b-49da-4b58-8fc3-60e32fb2a032 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:04.556702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:04.584205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727268b-49da-4b58-8fc3-60e32fb2a032 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:04.622711 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165d25a-97d6-41dd-b8ad-bee9e9c42b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:04.623555 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165d25a-97d6-41dd-b8ad-bee9e9c42b66 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:04.623833 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165d25a-97d6-41dd-b8ad-bee9e9c42b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:04.624104 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6165d25a-97d6-41dd-b8ad-bee9e9c42b66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:04.636317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165d25a-97d6-41dd-b8ad-bee9e9c42b66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:04.637834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6165d25a-97d6-41dd-b8ad-bee9e9c42b66 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:04.657951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-6165d25a-97d6-41dd-b8ad-bee9e9c42b66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:34:04.657951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:04.657951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:04.657951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:04.657951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:04.657951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:04.657951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:04.657951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:04.657951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:04.657951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:04.657951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:04.657951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:04.657951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:34:04.657951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:34:04.657951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:34:04.659179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:34:04.659179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:04.659179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:04.659179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:04.659179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:04.659179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:04.659179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:04.659179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:04.659179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:04.659179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:04.659179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:04.659179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:04.659179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:04.659179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:04.659179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:04.659179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:04.659179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:04.660072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:34:04.660072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:34:04.660072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:34:04.660072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:34:04.660581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 623/1251] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:34:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:04.667033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60fe91-3b35-4079-95fa-613d9759c843 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:04.790801 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60fe91-3b35-4079-95fa-613d9759c843 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:04.831500 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2091e-825f-42c7-80f6-bc48c7cbe299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:04.832235 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2091e-825f-42c7-80f6-bc48c7cbe299 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:04.832519 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2091e-825f-42c7-80f6-bc48c7cbe299 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:04.832890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2091e-825f-42c7-80f6-bc48c7cbe299 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:04.833943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2091e-825f-42c7-80f6-bc48c7cbe299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:04.834234 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2091e-825f-42c7-80f6-bc48c7cbe299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:04.836048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2091e-825f-42c7-80f6-bc48c7cbe299 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:04.851907 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 629/1252] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:34:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:04.857594 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165d25a-97d6-41dd-b8ad-bee9e9c42b66 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:04.965531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6165d25a-97d6-41dd-b8ad-bee9e9c42b66 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:04.994977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:05.004806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f04bc-e13d-4786-a917-645f5d8c9f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:05.005591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f04bc-e13d-4786-a917-645f5d8c9f1f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:05.005876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f04bc-e13d-4786-a917-645f5d8c9f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:05.006149 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f04bc-e13d-4786-a917-645f5d8c9f1f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:05.015862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f04bc-e13d-4786-a917-645f5d8c9f1f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:05.017586 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f04bc-e13d-4786-a917-645f5d8c9f1f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:05.035605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-6b3f04bc-e13d-4786-a917-645f5d8c9f1f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 05:34:05.035605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:05.035605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:05.035605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:05.035605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:05.035605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:05.035605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:05.035605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:05.035605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:05.035605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:05.035605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:05.035605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:05.035605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:34:05.035605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:34:05.035605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:34:05.035605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:05.037161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:05.038257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:05.038257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:05.038257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:34:05.038257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:34:05.038257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:34:05.038257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:34:05.038257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:34:05.038257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:34:05.038257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 05:34:05.038257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:34:05.038257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 624/1253] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:34:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:34:05.044052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2091e-825f-42c7-80f6-bc48c7cbe299 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:05.107931 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:05.163760 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2091e-825f-42c7-80f6-bc48c7cbe299 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:05.202573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c8eec-8879-4e64-9925-59a446ee8eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:05.203312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c8eec-8879-4e64-9925-59a446ee8eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:05.203630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c8eec-8879-4e64-9925-59a446ee8eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:05.203876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c8eec-8879-4e64-9925-59a446ee8eb4 None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:05.204708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c8eec-8879-4e64-9925-59a446ee8eb4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:05.205010 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c8eec-8879-4e64-9925-59a446ee8eb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:05.206508 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c8eec-8879-4e64-9925-59a446ee8eb4 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:05.224859 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 630/1254] 10.222.0.44 () {56 vars in 1143 bytes} [Thu Jul 21 05:34:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:05.231041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f04bc-e13d-4786-a917-645f5d8c9f1f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:05.344013 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f04bc-e13d-4786-a917-645f5d8c9f1f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:05.380234 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ff1df-e60f-4cbd-9f82-97915be6b5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:05.380784 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ff1df-e60f-4cbd-9f82-97915be6b5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:05.381064 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ff1df-e60f-4cbd-9f82-97915be6b5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:05.381294 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ff1df-e60f-4cbd-9f82-97915be6b5f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:05.447576 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:05.466129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ff1df-e60f-4cbd-9f82-97915be6b5f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:05.468213 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ff1df-e60f-4cbd-9f82-97915be6b5f2 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:05.593054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 625/1255] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:34:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 05:34:05.599062 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c8eec-8879-4e64-9925-59a446ee8eb4 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:05.719633 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c8eec-8879-4e64-9925-59a446ee8eb4 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:05.734359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:05.756490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d4310-c783-4af2-96fb-56d4bce72f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:05.757434 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d4310-c783-4af2-96fb-56d4bce72f71 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:05.757720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d4310-c783-4af2-96fb-56d4bce72f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:05.758027 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d4310-c783-4af2-96fb-56d4bce72f71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:05.768676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d4310-c783-4af2-96fb-56d4bce72f71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:05.770132 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d4310-c783-4af2-96fb-56d4bce72f71 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:05.788904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-363d4310-c783-4af2-96fb-56d4bce72f71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:34:05.788904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:05.788904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:05.788904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:05.788904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:05.788904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:05.788904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:05.788904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:05.788904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:05.788904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:05.788904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:05.788904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:05.788904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:34:05.788904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:34:05.788904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:34:05.789947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:34:05.789947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:05.789947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:05.789947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:05.789947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:05.789947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:05.789947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:05.789947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:05.789947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:05.789947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:05.789947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:05.789947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:05.789947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:05.789947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:05.789947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:05.789947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:05.789947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:05.795685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:34:05.795685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:34:05.795685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:34:05.795685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:34:05.795685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 631/1256] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:34:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 21 05:34:05.796294 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ff1df-e60f-4cbd-9f82-97915be6b5f2 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:05.891723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:05.906459 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ff1df-e60f-4cbd-9f82-97915be6b5f2 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:05.946512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a371fb-a13c-4232-913d-f88a077da966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:05.947133 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a371fb-a13c-4232-913d-f88a077da966 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:05.947415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a371fb-a13c-4232-913d-f88a077da966 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:05.947686 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a371fb-a13c-4232-913d-f88a077da966 None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:05.948411 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a371fb-a13c-4232-913d-f88a077da966 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:05.948717 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a371fb-a13c-4232-913d-f88a077da966 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:05.950086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a371fb-a13c-4232-913d-f88a077da966 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:05.962390 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 626/1257] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:05.968572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d4310-c783-4af2-96fb-56d4bce72f71 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:06.068947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d4310-c783-4af2-96fb-56d4bce72f71 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:06.103406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1625d60-9daf-47b8-b3c9-03a00f940852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:06.104306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1625d60-9daf-47b8-b3c9-03a00f940852 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:06.104586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1625d60-9daf-47b8-b3c9-03a00f940852 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:06.104885 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1625d60-9daf-47b8-b3c9-03a00f940852 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:06.117701 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1625d60-9daf-47b8-b3c9-03a00f940852 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:06.119194 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1625d60-9daf-47b8-b3c9-03a00f940852 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:06.133576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-b1625d60-9daf-47b8-b3c9-03a00f940852 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:34:06.133576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:06.133576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:06.133576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:06.133576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:06.133576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:06.133576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:06.133576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:06.133576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:06.133576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:06.133576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:06.133576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:06.133576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:34:06.133576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:34:06.133576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:34:06.134774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:34:06.134774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:06.134774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:06.134774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:06.134774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:06.134774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:06.134774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:06.134774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:06.134774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:06.134774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:06.134774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:06.134774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:06.134774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:06.134774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:06.134774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:06.134774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:06.134774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:06.140702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:34:06.140702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:34:06.140702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:34:06.140702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:34:06.140702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:34:06.140702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:34:06.140702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:34:06.140702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:34:06.140702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 632/1258] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:34:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:06.141310 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a371fb-a13c-4232-913d-f88a077da966 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:06.250575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a371fb-a13c-4232-913d-f88a077da966 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:06.285291 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1884a10-6db5-4b78-833a-650e6c519796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:06.286001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884a10-6db5-4b78-833a-650e6c519796 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:06.286677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884a10-6db5-4b78-833a-650e6c519796 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:06.286959 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884a10-6db5-4b78-833a-650e6c519796 None admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:06.287791 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1884a10-6db5-4b78-833a-650e6c519796 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:06.288097 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1884a10-6db5-4b78-833a-650e6c519796 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:06.289536 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1884a10-6db5-4b78-833a-650e6c519796 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:06.301949 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 627/1259] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:34:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:06.309401 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1625d60-9daf-47b8-b3c9-03a00f940852 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:06.337280 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:06.419104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1625d60-9daf-47b8-b3c9-03a00f940852 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:06.461307 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953ff7a-7081-4a1b-aa22-32c084ef2cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:06.462591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953ff7a-7081-4a1b-aa22-32c084ef2cfa None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:06.463254 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953ff7a-7081-4a1b-aa22-32c084ef2cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:06.463685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953ff7a-7081-4a1b-aa22-32c084ef2cfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:06.464628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953ff7a-7081-4a1b-aa22-32c084ef2cfa None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=03d868598e39448881b43d299253d821) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:06.464978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953ff7a-7081-4a1b-aa22-32c084ef2cfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:06.466470 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953ff7a-7081-4a1b-aa22-32c084ef2cfa None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:06.474691 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 633/1260] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:34:06 2022] GET /identity/v3/role_assignments?user.id=03d868598e39448881b43d299253d821&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:08.509781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88166ada-45dc-4730-8d10-8e034e97ce8b None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:08.510666 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88166ada-45dc-4730-8d10-8e034e97ce8b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:08.510666 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88166ada-45dc-4730-8d10-8e034e97ce8b None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:08.512823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 628/1261] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:34:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:34:08.520352 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeacc40-d207-4431-bc3c-a58327acf14c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:08.520810 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeacc40-d207-4431-bc3c-a58327acf14c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:08.520905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeacc40-d207-4431-bc3c-a58327acf14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:08.539964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-2eeacc40-d207-4431-bc3c-a58327acf14c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:34:08.541134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-2eeacc40-d207-4431-bc3c-a58327acf14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:34:08.610201 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 634/1262] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:34:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 21 05:34:08.644994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88166ada-45dc-4730-8d10-8e034e97ce8b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:08.658985 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:08.754705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88166ada-45dc-4730-8d10-8e034e97ce8b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:08.790036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66157d5f-ad93-4929-8137-98c9aaa01d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:08.790759 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66157d5f-ad93-4929-8137-98c9aaa01d20 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:08.791059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66157d5f-ad93-4929-8137-98c9aaa01d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:08.791338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66157d5f-ad93-4929-8137-98c9aaa01d20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:08.801779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66157d5f-ad93-4929-8137-98c9aaa01d20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:34:08.809233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-66157d5f-ad93-4929-8137-98c9aaa01d20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:34:08.809233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:08.809233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:08.809233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:08.809233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:08.809233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:08.809233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:08.809233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:08.809233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:08.809233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:08.809233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:08.809233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:08.809233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:34:08.809233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:34:08.809233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:34:08.809233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:34:08.810305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:34:08.810305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:34:08.810305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:34:08.810305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:34:08.810305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:08.810305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:08.810305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:08.810305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:08.810305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:08.810305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:08.810305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:08.810305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:08.810305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:08.810305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:08.810305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:08.810305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:08.811304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:08.811304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:08.811304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:08.811304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:08.811304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:34:08.811304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:34:08.811304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:34:08.811304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:34:08.811304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:34:08.811304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:34:08.811304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:34:08.811304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:34:08.811304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 629/1263] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:34:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:08.816922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeacc40-d207-4431-bc3c-a58327acf14c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:08.835402 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:08.929515 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeacc40-d207-4431-bc3c-a58327acf14c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:08.967334 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24607a44-e267-46b6-ae16-d5742574a538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:08.968018 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24607a44-e267-46b6-ae16-d5742574a538 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:08.968304 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24607a44-e267-46b6-ae16-d5742574a538 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:08.968577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24607a44-e267-46b6-ae16-d5742574a538 None admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:08.969328 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24607a44-e267-46b6-ae16-d5742574a538 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:08.969637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24607a44-e267-46b6-ae16-d5742574a538 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:08.971177 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24607a44-e267-46b6-ae16-d5742574a538 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:08.979904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 635/1264] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:34:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:08.986035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66157d5f-ad93-4929-8137-98c9aaa01d20 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:09.058313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:09.094406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66157d5f-ad93-4929-8137-98c9aaa01d20 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:09.131558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db882eb-8925-46be-ad9b-13964cc7287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:09.132355 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db882eb-8925-46be-ad9b-13964cc7287b None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:09.132649 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db882eb-8925-46be-ad9b-13964cc7287b None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:09.132922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db882eb-8925-46be-ad9b-13964cc7287b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:09.144564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db882eb-8925-46be-ad9b-13964cc7287b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:09.146023 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db882eb-8925-46be-ad9b-13964cc7287b None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:09.162957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-2db882eb-8925-46be-ad9b-13964cc7287b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:34:09.162957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:09.162957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:09.162957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:09.162957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:09.162957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:09.162957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:09.162957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:09.162957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:09.162957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:09.162957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:09.162957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:09.162957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:34:09.162957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:34:09.162957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:34:09.164080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:34:09.164080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:09.164080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:09.164080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:09.164080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:09.164080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:09.164080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:09.164080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:09.164080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:09.164080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:09.164080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:09.164080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:09.164080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:09.164080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:09.164080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:09.164080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:09.164080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:09.165121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:34:09.165121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:34:09.165121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:34:09.165121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:34:09.165121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 630/1265] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:34:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:09.170645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24607a44-e267-46b6-ae16-d5742574a538 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:09.255763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:09.280186 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24607a44-e267-46b6-ae16-d5742574a538 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:09.319703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb85da-fd26-408e-b8db-775dc7b3adfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:09.320291 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb85da-fd26-408e-b8db-775dc7b3adfa None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:09.320573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb85da-fd26-408e-b8db-775dc7b3adfa None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:09.320858 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb85da-fd26-408e-b8db-775dc7b3adfa None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:09.321573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb85da-fd26-408e-b8db-775dc7b3adfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:09.321872 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb85da-fd26-408e-b8db-775dc7b3adfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:09.323474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb85da-fd26-408e-b8db-775dc7b3adfa None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:09.336649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 636/1266] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:09.346210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db882eb-8925-46be-ad9b-13964cc7287b None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:09.483003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db882eb-8925-46be-ad9b-13964cc7287b None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:09.508517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:09.526286 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9b95e-2f4a-41ca-8b0d-278509993fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:09.527227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9b95e-2f4a-41ca-8b0d-278509993fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:09.527474 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9b95e-2f4a-41ca-8b0d-278509993fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:09.527748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9b95e-2f4a-41ca-8b0d-278509993fb3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:09.538162 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9b95e-2f4a-41ca-8b0d-278509993fb3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:09.539656 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9b95e-2f4a-41ca-8b0d-278509993fb3 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:09.553725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-b1e9b95e-2f4a-41ca-8b0d-278509993fb3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 05:34:09.553725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:09.553725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:09.553725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:09.553725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:09.553725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:09.553725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:09.553725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:09.553725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:09.553725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:09.553725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:09.553725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:09.553725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:34:09.553725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:34:09.553725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:34:09.553725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:09.554808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:09.556001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:09.556001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:09.556001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:34:09.556001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:34:09.556001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:34:09.556001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:34:09.556001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:34:09.556001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:34:09.556001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 05:34:09.556001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:34:09.556001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 631/1267] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:34:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:09.561053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb85da-fd26-408e-b8db-775dc7b3adfa None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:09.668232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb85da-fd26-408e-b8db-775dc7b3adfa None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:09.704013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7ed68-8525-47bd-8cd5-85da18e59f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:09.704693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7ed68-8525-47bd-8cd5-85da18e59f36 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:09.704982 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7ed68-8525-47bd-8cd5-85da18e59f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:09.705259 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7ed68-8525-47bd-8cd5-85da18e59f36 None admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:09.706098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7ed68-8525-47bd-8cd5-85da18e59f36 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:09.706411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7ed68-8525-47bd-8cd5-85da18e59f36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:09.707896 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7ed68-8525-47bd-8cd5-85da18e59f36 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:09.716730 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:09.727707 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 637/1268] 10.222.0.44 () {56 vars in 1143 bytes} [Thu Jul 21 05:34:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:09.733798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9b95e-2f4a-41ca-8b0d-278509993fb3 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:09.834498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9b95e-2f4a-41ca-8b0d-278509993fb3 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:09.868683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c587d-b698-41cd-b6ad-d874f9f56aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:09.869469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c587d-b698-41cd-b6ad-d874f9f56aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:09.869752 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c587d-b698-41cd-b6ad-d874f9f56aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:09.870026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c587d-b698-41cd-b6ad-d874f9f56aa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:09.880911 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c587d-b698-41cd-b6ad-d874f9f56aa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:09.882365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c587d-b698-41cd-b6ad-d874f9f56aa7 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:09.900415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-4c4c587d-b698-41cd-b6ad-d874f9f56aa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:34:09.900415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:09.900415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:09.900415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:09.900415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:09.900415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:09.900415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:09.900415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:09.900415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:09.900415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:09.900415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:09.900415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:09.900415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:34:09.900415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:34:09.900415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:34:09.901520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:34:09.901520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:09.901520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:09.901520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:09.901520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:09.901520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:09.901520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:09.901520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:09.901520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:09.901520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:09.901520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:09.901520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:09.901520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:09.901520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:09.901520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:09.901520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:09.901520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:09.902557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:34:09.902557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:34:09.902557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:34:09.902557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:34:09.902557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 632/1269] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:34:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:09.907807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7ed68-8525-47bd-8cd5-85da18e59f36 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:10.018906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7ed68-8525-47bd-8cd5-85da18e59f36 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:10.053539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644eaa3-b9cd-40d9-b72e-1331d6713277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:10.054202 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644eaa3-b9cd-40d9-b72e-1331d6713277 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:10.054483 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644eaa3-b9cd-40d9-b72e-1331d6713277 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:10.054764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644eaa3-b9cd-40d9-b72e-1331d6713277 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:10.055613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644eaa3-b9cd-40d9-b72e-1331d6713277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:10.055925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644eaa3-b9cd-40d9-b72e-1331d6713277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:10.057313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644eaa3-b9cd-40d9-b72e-1331d6713277 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:10.068660 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 638/1270] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:10.074658 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c587d-b698-41cd-b6ad-d874f9f56aa7 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:10.110367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:10.182949 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c587d-b698-41cd-b6ad-d874f9f56aa7 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:10.221650 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78c8e6-586d-4404-a177-ece8f36f22c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:10.222451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78c8e6-586d-4404-a177-ece8f36f22c3 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:10.222736 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78c8e6-586d-4404-a177-ece8f36f22c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:10.223002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78c8e6-586d-4404-a177-ece8f36f22c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:10.230236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78c8e6-586d-4404-a177-ece8f36f22c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:10.231728 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78c8e6-586d-4404-a177-ece8f36f22c3 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:10.245802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-da78c8e6-586d-4404-a177-ece8f36f22c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:34:10.245802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:10.245802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:10.245802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:10.245802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:10.245802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:10.245802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:10.245802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:10.245802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:10.245802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:10.245802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:10.245802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:10.245802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:34:10.245802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:34:10.245802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:34:10.246863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:34:10.246863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:10.246863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:10.246863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:10.246863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:10.246863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:10.246863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:10.246863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:10.246863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:10.246863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:10.246863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:10.246863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:10.246863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:10.246863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:10.246863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:10.246863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:10.246863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:10.247803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:34:10.247803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:34:10.247803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:34:10.247803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:34:10.247803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:34:10.247803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:34:10.247803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:34:10.247803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:34:10.247803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 633/1271] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:34:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:10.253118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644eaa3-b9cd-40d9-b72e-1331d6713277 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:10.317932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:10.364637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644eaa3-b9cd-40d9-b72e-1331d6713277 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:10.412305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd178f3c-4488-4129-aab9-332a6785b59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:10.412983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd178f3c-4488-4129-aab9-332a6785b59e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:10.413263 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd178f3c-4488-4129-aab9-332a6785b59e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:10.413542 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd178f3c-4488-4129-aab9-332a6785b59e None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:10.414312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd178f3c-4488-4129-aab9-332a6785b59e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:10.414616 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd178f3c-4488-4129-aab9-332a6785b59e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:10.416083 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd178f3c-4488-4129-aab9-332a6785b59e None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:10.430190 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 639/1272] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:34:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:10.438167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78c8e6-586d-4404-a177-ece8f36f22c3 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:10.542515 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:10.552402 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78c8e6-586d-4404-a177-ece8f36f22c3 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:10.589593 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ed2ac-1032-4c64-805b-45736c5141df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:10.590407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ed2ac-1032-4c64-805b-45736c5141df None admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:10.590687 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ed2ac-1032-4c64-805b-45736c5141df None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:10.590960 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ed2ac-1032-4c64-805b-45736c5141df None admin] PATH_INFO: `/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/03d868598e39448881b43d299253d821/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:10.593075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ed2ac-1032-4c64-805b-45736c5141df None admin] RBAC: Authorizing `identity:create_grant(project_id=74b6f27b8d9b44d191210a24a9e04041, user_id=03d868598e39448881b43d299253d821, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:10.594529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ed2ac-1032-4c64-805b-45736c5141df None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:10.608841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 634/1273] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jul 21 05:34:10 2022] PUT /identity/v3/projects/74b6f27b8d9b44d191210a24a9e04041/users/03d868598e39448881b43d299253d821/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:34:12.685800 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02479-52d9-480d-8e08-71334495a20b None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:12.686297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02479-52d9-480d-8e08-71334495a20b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:12.686397 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02479-52d9-480d-8e08-71334495a20b None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:12.687803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 640/1274] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:34:12.694206 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b63a0e-87ad-4918-b9a7-e8506d15e9b5 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:12.694965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b63a0e-87ad-4918-b9a7-e8506d15e9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:12.695140 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b63a0e-87ad-4918-b9a7-e8506d15e9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:12.722362 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-17b63a0e-87ad-4918-b9a7-e8506d15e9b5 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:34:12.723585 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-17b63a0e-87ad-4918-b9a7-e8506d15e9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:34:12.793165 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 635/1275] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:34:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:34:12.824114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf02479-52d9-480d-8e08-71334495a20b None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:12.934632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf02479-52d9-480d-8e08-71334495a20b None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:12.951872 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:12.977507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92b075-be1f-48c8-9794-9624944a112e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:12.978669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92b075-be1f-48c8-9794-9624944a112e None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:12.979066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92b075-be1f-48c8-9794-9624944a112e None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:12.979427 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92b075-be1f-48c8-9794-9624944a112e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:12.993297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe92b075-be1f-48c8-9794-9624944a112e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 21 05:34:13.000765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-fe92b075-be1f-48c8-9794-9624944a112e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:34:13.000765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:13.000765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:13.000765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:13.000765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:13.000765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:13.000765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:13.000765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:13.000765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:13.000765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:13.000765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:13.000765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:13.000765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 21 05:34:13.000765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 21 05:34:13.000765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 21 05:34:13.000765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 21 05:34:13.002005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 21 05:34:13.002005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 21 05:34:13.002005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 21 05:34:13.002005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 21 05:34:13.002005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:13.002005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:13.002005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:13.002005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:13.002005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:13.002005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:13.002005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:13.002005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:13.002005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:13.002005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:13.002005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:13.002005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:13.003165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:13.003165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:13.003165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:13.003165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:13.003165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 21 05:34:13.003165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 21 05:34:13.003165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 21 05:34:13.003165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 21 05:34:13.003165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 21 05:34:13.003165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 21 05:34:13.003165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 21 05:34:13.003165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:34:13.003165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 641/1276] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jul 21 05:34:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:13.008824 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b63a0e-87ad-4918-b9a7-e8506d15e9b5 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:13.096348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:13.122635 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b63a0e-87ad-4918-b9a7-e8506d15e9b5 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:13.158908 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f647f0-8a9c-406a-aaf5-02691482c491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:13.159614 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f647f0-8a9c-406a-aaf5-02691482c491 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:13.159899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f647f0-8a9c-406a-aaf5-02691482c491 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:13.160170 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f647f0-8a9c-406a-aaf5-02691482c491 None admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:13.160925 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f647f0-8a9c-406a-aaf5-02691482c491 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:13.161592 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f647f0-8a9c-406a-aaf5-02691482c491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:13.162980 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f647f0-8a9c-406a-aaf5-02691482c491 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:13.171474 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 636/1277] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jul 21 05:34:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:13.178157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92b075-be1f-48c8-9794-9624944a112e None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:13.283563 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92b075-be1f-48c8-9794-9624944a112e None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:13.321436 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2700a-d18e-4d41-94d5-56f12cbb8207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:13.322212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2700a-d18e-4d41-94d5-56f12cbb8207 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:13.322494 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2700a-d18e-4d41-94d5-56f12cbb8207 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:13.322762 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2700a-d18e-4d41-94d5-56f12cbb8207 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:13.333397 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2700a-d18e-4d41-94d5-56f12cbb8207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:13.334850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2700a-d18e-4d41-94d5-56f12cbb8207 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:13.351734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-5bd2700a-d18e-4d41-94d5-56f12cbb8207 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:34:13.351734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:13.351734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:13.351734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:13.351734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:13.351734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:13.351734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:13.351734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:13.351734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:13.351734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:13.351734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:13.351734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:13.351734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:34:13.351734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:34:13.351734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:34:13.352765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:34:13.352765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:13.352765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:13.352765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:13.352765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:13.352765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:13.352765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:13.352765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:13.352765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:13.352765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:13.352765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:13.352765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:13.352765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:13.352765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:13.352765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:13.352765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:13.352765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:13.354285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:34:13.354285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:34:13.354285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:34:13.354285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:34:13.354285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 642/1278] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:34:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:13.361419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f647f0-8a9c-406a-aaf5-02691482c491 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:13.471496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f647f0-8a9c-406a-aaf5-02691482c491 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:13.509948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e4cb3-af47-46cc-9cc3-529d0c0dc19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:13.510599 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e4cb3-af47-46cc-9cc3-529d0c0dc19b None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:13.510884 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e4cb3-af47-46cc-9cc3-529d0c0dc19b None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:13.511185 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e4cb3-af47-46cc-9cc3-529d0c0dc19b None admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:13.511912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e4cb3-af47-46cc-9cc3-529d0c0dc19b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:13.512221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e4cb3-af47-46cc-9cc3-529d0c0dc19b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:13.513591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e4cb3-af47-46cc-9cc3-529d0c0dc19b None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:13.525582 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 637/1279] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:13.532300 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2700a-d18e-4d41-94d5-56f12cbb8207 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:13.567195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:13.642064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2700a-d18e-4d41-94d5-56f12cbb8207 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:13.683245 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb85f9-fba3-427d-96e1-52a447c8b2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:13.684063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb85f9-fba3-427d-96e1-52a447c8b2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:13.684348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb85f9-fba3-427d-96e1-52a447c8b2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:13.684620 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb85f9-fba3-427d-96e1-52a447c8b2b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:13.692468 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb85f9-fba3-427d-96e1-52a447c8b2b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:13.693922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb85f9-fba3-427d-96e1-52a447c8b2b7 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:13.708370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-a7fb85f9-fba3-427d-96e1-52a447c8b2b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 21 05:34:13.708370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:13.708370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:13.708370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:13.708370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:13.708370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:13.708370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:13.708370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:13.708370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:13.708370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:13.708370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:13.708370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:13.708370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 21 05:34:13.708370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 21 05:34:13.708370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 21 05:34:13.708370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:13.709453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:13.710592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:13.710592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:13.710592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 21 05:34:13.710592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 21 05:34:13.710592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 21 05:34:13.710592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 21 05:34:13.710592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 21 05:34:13.710592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 21 05:34:13.710592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 21 05:34:13.710592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:34:13.714399 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 643/1280] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:34:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:13.715586 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e4cb3-af47-46cc-9cc3-529d0c0dc19b None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:13.720081 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:13.824291 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e4cb3-af47-46cc-9cc3-529d0c0dc19b None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:13.858897 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81df756-9c1d-49a3-95de-21f88796b665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:13.859581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81df756-9c1d-49a3-95de-21f88796b665 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:13.859861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81df756-9c1d-49a3-95de-21f88796b665 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:13.860133 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81df756-9c1d-49a3-95de-21f88796b665 None admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:13.860969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81df756-9c1d-49a3-95de-21f88796b665 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:13.861278 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81df756-9c1d-49a3-95de-21f88796b665 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:13.863039 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81df756-9c1d-49a3-95de-21f88796b665 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:13.879849 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 638/1281] 10.222.0.44 () {56 vars in 1143 bytes} [Thu Jul 21 05:34:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:13.889277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb85f9-fba3-427d-96e1-52a447c8b2b7 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:13.987393 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:13.997543 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb85f9-fba3-427d-96e1-52a447c8b2b7 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:14.032768 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db772c37-f25c-43b9-9347-8e3ea6e12fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:14.033521 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db772c37-f25c-43b9-9347-8e3ea6e12fbf None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:14.033808 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db772c37-f25c-43b9-9347-8e3ea6e12fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:14.034083 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db772c37-f25c-43b9-9347-8e3ea6e12fbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:14.105283 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db772c37-f25c-43b9-9347-8e3ea6e12fbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:14.106658 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db772c37-f25c-43b9-9347-8e3ea6e12fbf None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:14.206566 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:14.217709 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 644/1282] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jul 21 05:34:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 21 05:34:14.224197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81df756-9c1d-49a3-95de-21f88796b665 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:14.287301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:14.335288 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81df756-9c1d-49a3-95de-21f88796b665 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:14.371450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6caec6-9640-42f2-9b50-f4a11089287f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:14.372179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6caec6-9640-42f2-9b50-f4a11089287f None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:14.372459 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6caec6-9640-42f2-9b50-f4a11089287f None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:14.372736 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6caec6-9640-42f2-9b50-f4a11089287f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:14.383470 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6caec6-9640-42f2-9b50-f4a11089287f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:14.384921 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6caec6-9640-42f2-9b50-f4a11089287f None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:14.402575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-ad6caec6-9640-42f2-9b50-f4a11089287f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:34:14.402575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:14.402575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:14.402575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:14.402575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:14.402575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:14.402575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:14.402575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:14.402575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:14.402575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:14.402575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:14.402575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:14.402575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 21 05:34:14.402575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 21 05:34:14.402575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 21 05:34:14.403652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 21 05:34:14.403652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:14.403652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:14.403652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:14.403652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:14.403652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:14.403652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:14.403652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:14.403652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:14.403652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:14.403652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:14.403652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:14.403652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:14.403652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:14.403652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:14.403652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:14.403652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:14.404815 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 21 05:34:14.404815 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 21 05:34:14.404815 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 21 05:34:14.404815 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:34:14.404815 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 639/1283] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:34:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:14.412458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db772c37-f25c-43b9-9347-8e3ea6e12fbf None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:14.520397 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db772c37-f25c-43b9-9347-8e3ea6e12fbf None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:14.556059 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2549de-16c5-48e5-b619-037078e45f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:14.556722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2549de-16c5-48e5-b619-037078e45f39 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:14.557008 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2549de-16c5-48e5-b619-037078e45f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:14.557282 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2549de-16c5-48e5-b619-037078e45f39 None admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:14.558001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2549de-16c5-48e5-b619-037078e45f39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:14.558312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2549de-16c5-48e5-b619-037078e45f39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:14.559720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2549de-16c5-48e5-b619-037078e45f39 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:14.571234 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 645/1284] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jul 21 05:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:14.577284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6caec6-9640-42f2-9b50-f4a11089287f None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:14.683266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6caec6-9640-42f2-9b50-f4a11089287f None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:14.697956 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:14.720294 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf569fc-30fc-40df-a3e2-31ecc5400b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:14.721147 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf569fc-30fc-40df-a3e2-31ecc5400b1a None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:14.721429 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf569fc-30fc-40df-a3e2-31ecc5400b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:14.721697 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf569fc-30fc-40df-a3e2-31ecc5400b1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:14.728389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf569fc-30fc-40df-a3e2-31ecc5400b1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:14.729838 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf569fc-30fc-40df-a3e2-31ecc5400b1a None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:14.743343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application [None req-8cf569fc-30fc-40df-a3e2-31ecc5400b1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:34:14.743343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:14.743343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:14.743343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:14.743343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:14.743343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:14.743343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:14.743343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:14.743343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:14.743343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:14.743343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:14.743343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:14.743343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 21 05:34:14.743343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 21 05:34:14.743343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 21 05:34:14.744348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 21 05:34:14.744348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:14.744348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:14.744348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:14.744348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:14.744348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:14.744348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:14.744348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:14.744348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:14.744348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:14.744348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:14.744348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:14.744348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:14.744348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:14.744348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:14.744348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:14.744348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:14.745438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 21 05:34:14.745438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 21 05:34:14.745438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 21 05:34:14.745438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 21 05:34:14.745438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 21 05:34:14.745438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 21 05:34:14.745438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 21 05:34:14.745438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: ERROR keystone.server.flask.application Jul 21 05:34:14.747186 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 640/1285] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jul 21 05:34:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:14.751070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2549de-16c5-48e5-b619-037078e45f39 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:14.802576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:14.863531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2549de-16c5-48e5-b619-037078e45f39 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:14.898752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8bd01-cd69-4c3a-a7d3-328e17529b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:14.899442 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8bd01-cd69-4c3a-a7d3-328e17529b77 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:14.899722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8bd01-cd69-4c3a-a7d3-328e17529b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:14.899996 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8bd01-cd69-4c3a-a7d3-328e17529b77 None admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:14.900784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8bd01-cd69-4c3a-a7d3-328e17529b77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:14.901097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8bd01-cd69-4c3a-a7d3-328e17529b77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:14.902551 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8bd01-cd69-4c3a-a7d3-328e17529b77 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:14.914962 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 646/1286] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jul 21 05:34:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:14.921856 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf569fc-30fc-40df-a3e2-31ecc5400b1a None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:15.032487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf569fc-30fc-40df-a3e2-31ecc5400b1a None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:15.067093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4860e-5e31-4535-a849-119860f7f27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:15.067733 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4860e-5e31-4535-a849-119860f7f27e None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:15.068010 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4860e-5e31-4535-a849-119860f7f27e None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:15.068279 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4860e-5e31-4535-a849-119860f7f27e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:15.069101 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4860e-5e31-4535-a849-119860f7f27e None admin] RBAC: Adding query filter params (role.id=6a5988fad9e245eba3c55eba854db4e8, scope.project.id=74b6f27b8d9b44d191210a24a9e04041, user.id=03d868598e39448881b43d299253d821) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:15.069408 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4860e-5e31-4535-a849-119860f7f27e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:15.070829 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4860e-5e31-4535-a849-119860f7f27e None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:15.076745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 641/1287] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jul 21 05:34:14 2022] GET /identity/v3/role_assignments?user.id=03d868598e39448881b43d299253d821&scope.project.id=74b6f27b8d9b44d191210a24a9e04041&role.id=6a5988fad9e245eba3c55eba854db4e8 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:17.080899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e4819-df56-4d0a-846a-89933bb59c47 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:17.081441 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e4819-df56-4d0a-846a-89933bb59c47 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:17.081441 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e4819-df56-4d0a-846a-89933bb59c47 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:17.082744 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 647/1288] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:34:17.089093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f5f7d-b6b5-4cd1-b37a-fb1840c1b700 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:17.089838 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f5f7d-b6b5-4cd1-b37a-fb1840c1b700 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:17.089974 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f5f7d-b6b5-4cd1-b37a-fb1840c1b700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:17.111481 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-280f5f7d-b6b5-4cd1-b37a-fb1840c1b700 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:34:17.112677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-280f5f7d-b6b5-4cd1-b37a-fb1840c1b700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:34:17.145412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:17.192711 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 642/1289] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:34:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:34:17.226540 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e4819-df56-4d0a-846a-89933bb59c47 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:17.337246 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e4819-df56-4d0a-846a-89933bb59c47 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:17.348030 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:17.374510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14bc0fd-93de-4530-88be-9286e846ff07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:17.375347 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14bc0fd-93de-4530-88be-9286e846ff07 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:17.375628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14bc0fd-93de-4530-88be-9286e846ff07 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:17.375909 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14bc0fd-93de-4530-88be-9286e846ff07 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:17.376557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14bc0fd-93de-4530-88be-9286e846ff07 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:17.377977 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14bc0fd-93de-4530-88be-9286e846ff07 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:17.389952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-e14bc0fd-93de-4530-88be-9286e846ff07 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 05:34:17.389952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:17.389952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:34:17.389952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:34:17.389952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:34:17.389952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:34:17.389952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:34:17.389952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:34:17.389952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 05:34:17.389952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:34:17.389952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:34:17.389952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:34:17.389952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:17.389952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:17.389952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:17.389952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:17.389952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:17.391175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:17.391175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:17.391175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:17.391175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:17.391175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:17.391175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:17.391175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:34:17.391175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:34:17.391175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:34:17.391175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:34:17.391175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:17.391175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:17.391175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:17.391175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:17.391175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:17.391175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:17.392322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:17.392322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:17.392322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:17.392322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:17.392322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:17.392322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:17.392322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:17.392322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:17.392322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:17.392322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:17.392322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:34:17.392322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:34:17.392322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 05:34:17.392322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:34:17.395181 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 648/1290] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jul 21 05:34:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:17.397760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f5f7d-b6b5-4cd1-b37a-fb1840c1b700 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:17.524264 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f5f7d-b6b5-4cd1-b37a-fb1840c1b700 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:17.561251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb9c33-c86a-4fec-a7a5-af09b88343f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:17.561917 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb9c33-c86a-4fec-a7a5-af09b88343f9 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:17.562186 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb9c33-c86a-4fec-a7a5-af09b88343f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:17.562509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb9c33-c86a-4fec-a7a5-af09b88343f9 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:17.563242 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb9c33-c86a-4fec-a7a5-af09b88343f9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:17.563552 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb9c33-c86a-4fec-a7a5-af09b88343f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:17.564921 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb9c33-c86a-4fec-a7a5-af09b88343f9 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:17.569980 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 643/1291] 10.222.0.44 () {56 vars in 1116 bytes} [Thu Jul 21 05:34:17 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:17.576100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14bc0fd-93de-4530-88be-9286e846ff07 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:17.679074 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14bc0fd-93de-4530-88be-9286e846ff07 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:17.713232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb88bf1-122f-4607-8ae9-36a7bbc49c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:17.713892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb88bf1-122f-4607-8ae9-36a7bbc49c41 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:17.714170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb88bf1-122f-4607-8ae9-36a7bbc49c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:17.714442 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb88bf1-122f-4607-8ae9-36a7bbc49c41 None admin] PATH_INFO: `/v3/services` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:17.715169 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb88bf1-122f-4607-8ae9-36a7bbc49c41 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:17.715478 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb88bf1-122f-4607-8ae9-36a7bbc49c41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:17.716842 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb88bf1-122f-4607-8ae9-36a7bbc49c41 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:17.721603 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 649/1292] 10.222.0.44 () {56 vars in 1116 bytes} [Thu Jul 21 05:34:17 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:34:19.745052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff6cf2-5ece-45c0-b40c-0d78398fe540 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:19.745883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff6cf2-5ece-45c0-b40c-0d78398fe540 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:19.745976 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff6cf2-5ece-45c0-b40c-0d78398fe540 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:19.748176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 644/1293] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:34:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:34:19.753559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6c70f-1c42-4a6d-9f4c-7cd317ad35a7 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:19.753996 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6c70f-1c42-4a6d-9f4c-7cd317ad35a7 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:19.754097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6c70f-1c42-4a6d-9f4c-7cd317ad35a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:19.772806 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-5ea6c70f-1c42-4a6d-9f4c-7cd317ad35a7 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:34:19.773985 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5ea6c70f-1c42-4a6d-9f4c-7cd317ad35a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:34:19.804076 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:19.843571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 650/1294] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:34:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:34:19.867255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff6cf2-5ece-45c0-b40c-0d78398fe540 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:19.912130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:19.995710 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff6cf2-5ece-45c0-b40c-0d78398fe540 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:20.030467 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a432ef-7564-47af-9526-95749ae65903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:20.031154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a432ef-7564-47af-9526-95749ae65903 None admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:20.031434 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a432ef-7564-47af-9526-95749ae65903 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:20.031703 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a432ef-7564-47af-9526-95749ae65903 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:20.032461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a432ef-7564-47af-9526-95749ae65903 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:20.034467 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a432ef-7564-47af-9526-95749ae65903 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:20.049982 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 645/1295] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jul 21 05:34:19 2022] POST /identity/v3/services => generated 246 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:34:22.162141 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc29e2-0bb2-4a46-9ded-7ce8b1068d0b None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:22.162992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc29e2-0bb2-4a46-9ded-7ce8b1068d0b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:22.163192 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc29e2-0bb2-4a46-9ded-7ce8b1068d0b None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:22.165321 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 651/1296] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:34:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:34:22.174042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc32a9-0a6e-49b3-814b-02f4ba017970 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:22.174758 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc32a9-0a6e-49b3-814b-02f4ba017970 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:22.174882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc32a9-0a6e-49b3-814b-02f4ba017970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:22.198147 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-a9dc32a9-0a6e-49b3-814b-02f4ba017970 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:34:22.199386 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a9dc32a9-0a6e-49b3-814b-02f4ba017970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:34:22.283885 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 646/1297] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:34:22 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:34:22.307560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc29e2-0bb2-4a46-9ded-7ce8b1068d0b None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:22.411920 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc29e2-0bb2-4a46-9ded-7ce8b1068d0b None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:22.452784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d5f43-d894-4389-9801-57db76ed2684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:22.453744 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d5f43-d894-4389-9801-57db76ed2684 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:22.454059 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d5f43-d894-4389-9801-57db76ed2684 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:22.454415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d5f43-d894-4389-9801-57db76ed2684 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:22.455360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d5f43-d894-4389-9801-57db76ed2684 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:22.457133 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d5f43-d894-4389-9801-57db76ed2684 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:22.473645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-ed1d5f43-d894-4389-9801-57db76ed2684 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 05:34:22.473645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:22.473645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:34:22.473645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:34:22.473645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:34:22.473645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:34:22.473645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:34:22.473645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:34:22.473645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 05:34:22.473645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:34:22.473645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:34:22.473645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:34:22.473645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:22.473645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:22.473645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:22.473645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:22.473645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:22.474836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:22.474836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:22.474836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:22.474836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:22.474836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:22.474836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:22.474836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:34:22.474836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:34:22.474836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:34:22.474836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:34:22.474836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:22.474836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:22.474836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:22.474836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:22.474836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:22.474836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:22.475819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:22.475819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:22.475819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:22.475819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:22.475819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:22.475819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:22.475819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:22.475819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:22.475819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:22.475819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:22.475819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:34:22.475819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:34:22.475819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 05:34:22.475819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:34:22.475819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 652/1298] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jul 21 05:34:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:22.481622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc32a9-0a6e-49b3-814b-02f4ba017970 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:22.503037 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:22.589677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc32a9-0a6e-49b3-814b-02f4ba017970 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:22.627119 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52621fce-b1bf-4341-a7e8-8e6efceb5259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:22.627780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52621fce-b1bf-4341-a7e8-8e6efceb5259 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:22.628058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52621fce-b1bf-4341-a7e8-8e6efceb5259 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:22.628330 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52621fce-b1bf-4341-a7e8-8e6efceb5259 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:22.629029 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52621fce-b1bf-4341-a7e8-8e6efceb5259 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:22.629338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52621fce-b1bf-4341-a7e8-8e6efceb5259 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:22.630715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52621fce-b1bf-4341-a7e8-8e6efceb5259 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:22.636006 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 647/1299] 10.222.0.44 () {56 vars in 1116 bytes} [Thu Jul 21 05:34:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:22.642955 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d5f43-d894-4389-9801-57db76ed2684 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:22.678040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:22.753829 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d5f43-d894-4389-9801-57db76ed2684 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:22.794205 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9ec0f-33d2-4783-a93d-7d6056fa6edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:22.794856 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9ec0f-33d2-4783-a93d-7d6056fa6edd None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:22.795171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9ec0f-33d2-4783-a93d-7d6056fa6edd None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:22.795440 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9ec0f-33d2-4783-a93d-7d6056fa6edd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:22.796221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9ec0f-33d2-4783-a93d-7d6056fa6edd None admin] RBAC: Adding query filter params (interface=public, service_id=85c7d203e3394960aceb8259e9781a26, region_id=RegionOne) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:22.796529 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9ec0f-33d2-4783-a93d-7d6056fa6edd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:22.797900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e9ec0f-33d2-4783-a93d-7d6056fa6edd None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:22.803046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 653/1300] 10.222.0.44 () {56 vars in 1251 bytes} [Thu Jul 21 05:34:22 2022] GET /identity/v3/endpoints?service_id=85c7d203e3394960aceb8259e9781a26&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:22.809120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52621fce-b1bf-4341-a7e8-8e6efceb5259 None admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:22.895902 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:22.915023 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52621fce-b1bf-4341-a7e8-8e6efceb5259 None admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:22.949576 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ea624-988c-4ded-accd-917b1a969027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:22.950233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ea624-988c-4ded-accd-917b1a969027 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:22.950511 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ea624-988c-4ded-accd-917b1a969027 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:22.950780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ea624-988c-4ded-accd-917b1a969027 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:22.951493 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ea624-988c-4ded-accd-917b1a969027 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:22.952855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ea624-988c-4ded-accd-917b1a969027 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:22.959894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 648/1301] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jul 21 05:34:22 2022] GET /identity/v3/services => generated 2191 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:34:24.936203 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29022b01-7d8e-4447-9dc5-49ca89da1e8e None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:24.936738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29022b01-7d8e-4447-9dc5-49ca89da1e8e None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:24.936738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29022b01-7d8e-4447-9dc5-49ca89da1e8e None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:24.938868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 654/1302] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:34:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:34:24.947745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bdc54-1b1b-4642-b02e-e3adacc9c08c None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:24.948134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bdc54-1b1b-4642-b02e-e3adacc9c08c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:24.948398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bdc54-1b1b-4642-b02e-e3adacc9c08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:24.969458 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-4e8bdc54-1b1b-4642-b02e-e3adacc9c08c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:34:24.970645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4e8bdc54-1b1b-4642-b02e-e3adacc9c08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:34:25.042424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 649/1303] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:34:24 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 21 05:34:25.068036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29022b01-7d8e-4447-9dc5-49ca89da1e8e None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:25.181216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:25.191066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29022b01-7d8e-4447-9dc5-49ca89da1e8e None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:25.225647 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc999f-24c2-4b28-b10a-6e90ebe79763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:25.226454 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc999f-24c2-4b28-b10a-6e90ebe79763 None admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:25.226743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc999f-24c2-4b28-b10a-6e90ebe79763 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:25.227069 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc999f-24c2-4b28-b10a-6e90ebe79763 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:25.227698 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc999f-24c2-4b28-b10a-6e90ebe79763 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:25.229077 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc999f-24c2-4b28-b10a-6e90ebe79763 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:25.241009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application [None req-c5bc999f-24c2-4b28-b10a-6e90ebe79763 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 05:34:25.241009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:25.241009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 21 05:34:25.241009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 21 05:34:25.241009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 21 05:34:25.241009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 21 05:34:25.241009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 21 05:34:25.241009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:34:25.241009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 05:34:25.241009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:34:25.241009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 21 05:34:25.241009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:34:25.241009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 21 05:34:25.241009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 21 05:34:25.241009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 21 05:34:25.241009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 21 05:34:25.241009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 21 05:34:25.242147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 21 05:34:25.242147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 21 05:34:25.242147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 21 05:34:25.242147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 21 05:34:25.242147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 21 05:34:25.242147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 21 05:34:25.242147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 21 05:34:25.242147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 21 05:34:25.242147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 21 05:34:25.242147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 21 05:34:25.242147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 21 05:34:25.242147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 21 05:34:25.242147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 21 05:34:25.242147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 21 05:34:25.242147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 21 05:34:25.242147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.get_or_create( Jul 21 05:34:25.243230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 21 05:34:25.243230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application with Lock( Jul 21 05:34:25.243230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 21 05:34:25.243230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self._enter() Jul 21 05:34:25.243230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 21 05:34:25.243230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 21 05:34:25.243230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 21 05:34:25.243230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application return self.creator() Jul 21 05:34:25.243230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 21 05:34:25.243230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application created_value = creator( Jul 21 05:34:25.243230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 21 05:34:25.243230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 21 05:34:25.243230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 21 05:34:25.243230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: ERROR keystone.server.flask.application Jul 21 05:34:25.243230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 655/1304] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jul 21 05:34:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 21 05:34:25.248545 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bdc54-1b1b-4642-b02e-e3adacc9c08c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:25.331502 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:25.356090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bdc54-1b1b-4642-b02e-e3adacc9c08c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:25.390671 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c4e99-cf47-4035-b816-32413a7224c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:25.391366 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c4e99-cf47-4035-b816-32413a7224c0 None admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:25.391652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c4e99-cf47-4035-b816-32413a7224c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:25.391918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c4e99-cf47-4035-b816-32413a7224c0 None admin] PATH_INFO: `/v3/services` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:25.392689 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c4e99-cf47-4035-b816-32413a7224c0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:34:25.392998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c4e99-cf47-4035-b816-32413a7224c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:25.394370 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c4e99-cf47-4035-b816-32413a7224c0 None admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:25.399596 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 650/1305] 10.222.0.44 () {56 vars in 1116 bytes} [Thu Jul 21 05:34:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:34:25.406058 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc999f-24c2-4b28-b10a-6e90ebe79763 None admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:34:25.516504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc999f-24c2-4b28-b10a-6e90ebe79763 None admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:34:25.554196 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc4782f-59f7-4e7d-8836-0959db7fd4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:34:25.554864 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4782f-59f7-4e7d-8836-0959db7fd4c8 None admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:34:25.555259 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4782f-59f7-4e7d-8836-0959db7fd4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:34:25.555536 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc4782f-59f7-4e7d-8836-0959db7fd4c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:34:25.556282 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4782f-59f7-4e7d-8836-0959db7fd4c8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:34:25.557664 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc4782f-59f7-4e7d-8836-0959db7fd4c8 None admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:34:25.578336 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 656/1306] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jul 21 05:34:25 2022] POST /identity/v3/endpoints => generated 331 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:34:35.396474 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=30 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:34:35.573636 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:28.298810 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cfd6d-43a4-4c6d-bdb5-4317b8fbd7ee None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:28.299734 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cfd6d-43a4-4c6d-bdb5-4317b8fbd7ee None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:28.300417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463cfd6d-43a4-4c6d-bdb5-4317b8fbd7ee None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:28.307803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84ca65-8e63-40ca-bc40-72bdf281d6ba None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:28.308712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84ca65-8e63-40ca-bc40-72bdf281d6ba None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:28.309211 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84ca65-8e63-40ca-bc40-72bdf281d6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:28.311422 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 651/1307] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:35:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:35:28.425825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-db84ca65-8e63-40ca-bc40-72bdf281d6ba None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:35:28.427489 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-db84ca65-8e63-40ca-bc40-72bdf281d6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:28.575130 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 657/1308] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:35:28.625804 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac7bb1-34d4-41f4-a027-e26a9249a577 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:28.626577 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac7bb1-34d4-41f4-a027-e26a9249a577 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:28.627003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac7bb1-34d4-41f4-a027-e26a9249a577 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:28.628677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 652/1309] 10.222.0.44 () {54 vars in 965 bytes} [Thu Jul 21 05:35:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:35:28.638282 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f851e4-a22a-4aee-b398-2432f87500f8 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:28.638664 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f851e4-a22a-4aee-b398-2432f87500f8 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:28.639029 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f851e4-a22a-4aee-b398-2432f87500f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:28.815775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-b7f851e4-a22a-4aee-b398-2432f87500f8 None None] MFA Rules not processed for user `e07508a9fa9146528620b7f8abec8549`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:35:28.818614 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b7f851e4-a22a-4aee-b398-2432f87500f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:28.845024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:28.912572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 658/1310] 10.222.0.44 () {58 vars in 1062 bytes} [Thu Jul 21 05:35:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:35:28.921749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac7bb1-34d4-41f4-a027-e26a9249a577 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:35:28.923953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2fac7bb1-34d4-41f4-a027-e26a9249a577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:29.076802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac7bb1-34d4-41f4-a027-e26a9249a577 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:35:29.182718 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d097fae-f64a-449b-a898-af4604e14fd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:35:29.183531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d097fae-f64a-449b-a898-af4604e14fd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:29.183884 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d097fae-f64a-449b-a898-af4604e14fd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:29.184228 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d097fae-f64a-449b-a898-af4604e14fd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:29.185964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8d097fae-f64a-449b-a898-af4604e14fd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:29.253151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:29.320971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d097fae-f64a-449b-a898-af4604e14fd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:35:29.323319 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d097fae-f64a-449b-a898-af4604e14fd8 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:35:29.345801 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 653/1311] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:35:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:35:31.682045 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6790768-ea14-4fa3-b7dc-5a3c1de0a8af None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:31.682541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6790768-ea14-4fa3-b7dc-5a3c1de0a8af None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:31.682634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6790768-ea14-4fa3-b7dc-5a3c1de0a8af None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:31.684041 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 659/1312] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:35:31.689692 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed16d6-830f-456e-85c3-06c154037a59 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:31.690157 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed16d6-830f-456e-85c3-06c154037a59 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:31.690240 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed16d6-830f-456e-85c3-06c154037a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:31.770752 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-0eed16d6-830f-456e-85c3-06c154037a59 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:35:31.771869 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-0eed16d6-830f-456e-85c3-06c154037a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:31.881266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 654/1313] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:35:31.936706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea90ae-3c33-492b-9760-8d2818ccea37 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:31.936980 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea90ae-3c33-492b-9760-8d2818ccea37 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:31.937245 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea90ae-3c33-492b-9760-8d2818ccea37 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:31.938674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 660/1314] 10.222.0.44 () {54 vars in 965 bytes} [Thu Jul 21 05:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:35:31.945358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789db396-ad17-4f98-8c6c-fd268e95a763 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:31.945649 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789db396-ad17-4f98-8c6c-fd268e95a763 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:31.945929 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789db396-ad17-4f98-8c6c-fd268e95a763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:31.957003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:32.024236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-789db396-ad17-4f98-8c6c-fd268e95a763 None None] MFA Rules not processed for user `e07508a9fa9146528620b7f8abec8549`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:35:32.025374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-789db396-ad17-4f98-8c6c-fd268e95a763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:32.104267 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 655/1315] 10.222.0.44 () {58 vars in 1062 bytes} [Thu Jul 21 05:35:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:35:32.113762 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea90ae-3c33-492b-9760-8d2818ccea37 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:35:32.115338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-abea90ae-3c33-492b-9760-8d2818ccea37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:32.219573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea90ae-3c33-492b-9760-8d2818ccea37 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:35:32.294783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:32.297570 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90b4c3-b13a-46b0-ba14-458d2ad08247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:35:32.298249 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90b4c3-b13a-46b0-ba14-458d2ad08247 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:32.298536 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90b4c3-b13a-46b0-ba14-458d2ad08247 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:32.298813 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90b4c3-b13a-46b0-ba14-458d2ad08247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:32.300837 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9f90b4c3-b13a-46b0-ba14-458d2ad08247 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:32.408809 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f90b4c3-b13a-46b0-ba14-458d2ad08247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:35:32.410242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f90b4c3-b13a-46b0-ba14-458d2ad08247 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:35:32.428890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 661/1316] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:35:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:35:34.973029 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250364e5-e1b0-4d8a-91e4-9abe79cec6cc None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:34.973873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250364e5-e1b0-4d8a-91e4-9abe79cec6cc None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:34.973873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250364e5-e1b0-4d8a-91e4-9abe79cec6cc None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:34.976149 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 656/1317] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:35:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:35:34.981539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a479f84-3915-4726-a2f0-b0d590824eaf None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:34.981960 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a479f84-3915-4726-a2f0-b0d590824eaf None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:34.982051 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a479f84-3915-4726-a2f0-b0d590824eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:35.051170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-7a479f84-3915-4726-a2f0-b0d590824eaf None None] MFA Rules not processed for user `c23f5930d01c4d6f819cc9308da0b14e`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:35:35.052351 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7a479f84-3915-4726-a2f0-b0d590824eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:35.104630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:35.149479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 662/1318] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:35:34 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:35:35.194823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250364e5-e1b0-4d8a-91e4-9abe79cec6cc None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:35:35.197443 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-250364e5-e1b0-4d8a-91e4-9abe79cec6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:35.330960 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250364e5-e1b0-4d8a-91e4-9abe79cec6cc None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:35:35.335180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:35.410678 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1049a19-81d6-4827-b8be-af5b44157b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:35:35.411365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1049a19-81d6-4827-b8be-af5b44157b04 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:35.411650 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1049a19-81d6-4827-b8be-af5b44157b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:35.411924 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1049a19-81d6-4827-b8be-af5b44157b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:35.413566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d1049a19-81d6-4827-b8be-af5b44157b04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:35.523339 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1049a19-81d6-4827-b8be-af5b44157b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:35:35.524798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1049a19-81d6-4827-b8be-af5b44157b04 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:35:35.543602 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 657/1319] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:35:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:35:38.158251 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d07ef-71cc-4ccc-9c4c-a6495a498fa3 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:38.159200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d07ef-71cc-4ccc-9c4c-a6495a498fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:38.159200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d07ef-71cc-4ccc-9c4c-a6495a498fa3 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:38.161672 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 663/1320] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:35:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:35:38.167063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b1b14-e6de-490a-a2a8-facdcd75a278 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:38.167487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b1b14-e6de-490a-a2a8-facdcd75a278 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:38.167606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b1b14-e6de-490a-a2a8-facdcd75a278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:38.207393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:38.242541 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-614b1b14-e6de-490a-a2a8-facdcd75a278 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:35:38.243864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-614b1b14-e6de-490a-a2a8-facdcd75a278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:38.343619 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 658/1321] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:35:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:35:38.386063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d07ef-71cc-4ccc-9c4c-a6495a498fa3 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:35:38.388538 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-739d07ef-71cc-4ccc-9c4c-a6495a498fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:38.510956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d07ef-71cc-4ccc-9c4c-a6495a498fa3 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:35:38.592180 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4d905-1bf1-4d2d-9a5d-529b94adc407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:35:38.592835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4d905-1bf1-4d2d-9a5d-529b94adc407 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:38.593121 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4d905-1bf1-4d2d-9a5d-529b94adc407 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:38.593392 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4d905-1bf1-4d2d-9a5d-529b94adc407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:38.595071 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7ae4d905-1bf1-4d2d-9a5d-529b94adc407 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:38.602691 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:38.718882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4d905-1bf1-4d2d-9a5d-529b94adc407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:35:38.720373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4d905-1bf1-4d2d-9a5d-529b94adc407 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:35:38.739165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 664/1322] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:35:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:35:41.662749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf240a-fa70-49b5-b628-170aab8e64e6 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:41.663158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf240a-fa70-49b5-b628-170aab8e64e6 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:41.663267 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf240a-fa70-49b5-b628-170aab8e64e6 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:41.664593 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 659/1323] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:35:41.670047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb696fae-3d14-472f-a337-49722c23c215 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:41.670486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb696fae-3d14-472f-a337-49722c23c215 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:41.670587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb696fae-3d14-472f-a337-49722c23c215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:41.735829 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-fb696fae-3d14-472f-a337-49722c23c215 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:35:41.737018 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-fb696fae-3d14-472f-a337-49722c23c215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:41.800398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:41.844055 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 665/1324] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:35:41.880766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf240a-fa70-49b5-b628-170aab8e64e6 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:35:41.882884 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-21bf240a-fa70-49b5-b628-170aab8e64e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:41.992466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf240a-fa70-49b5-b628-170aab8e64e6 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:35:42.007999 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:42.072903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1344ad-2c56-4004-b2bc-3b9d1f1d613d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:35:42.073466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1344ad-2c56-4004-b2bc-3b9d1f1d613d service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:42.073747 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1344ad-2c56-4004-b2bc-3b9d1f1d613d service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:42.074020 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1344ad-2c56-4004-b2bc-3b9d1f1d613d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:42.075663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-9d1344ad-2c56-4004-b2bc-3b9d1f1d613d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:42.186010 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1344ad-2c56-4004-b2bc-3b9d1f1d613d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:35:42.187506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1344ad-2c56-4004-b2bc-3b9d1f1d613d service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:35:42.205771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 660/1325] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:35:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:35:44.261349 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695e92f-8efd-427c-974f-445b735ccd2e None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:44.261897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695e92f-8efd-427c-974f-445b735ccd2e None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:44.261897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695e92f-8efd-427c-974f-445b735ccd2e None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:44.263202 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 666/1326] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:35:44.268542 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbee9d6-3897-49fe-97c7-672462a7da23 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:44.268990 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbee9d6-3897-49fe-97c7-672462a7da23 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:44.269082 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbee9d6-3897-49fe-97c7-672462a7da23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:44.317532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:44.336742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-0cbee9d6-3897-49fe-97c7-672462a7da23 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:35:44.337950 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-0cbee9d6-3897-49fe-97c7-672462a7da23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:44.436483 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 661/1327] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:35:44.473628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695e92f-8efd-427c-974f-445b735ccd2e None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:35:44.475219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d695e92f-8efd-427c-974f-445b735ccd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:44.581250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695e92f-8efd-427c-974f-445b735ccd2e None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:35:44.657488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ad272-8ecc-4cf7-86f5-a6c4b7583a8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:35:44.658177 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ad272-8ecc-4cf7-86f5-a6c4b7583a8b service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:44.658429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ad272-8ecc-4cf7-86f5-a6c4b7583a8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:44.658732 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ad272-8ecc-4cf7-86f5-a6c4b7583a8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:44.669024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-de4ad272-8ecc-4cf7-86f5-a6c4b7583a8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:44.687610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:44.781450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ad272-8ecc-4cf7-86f5-a6c4b7583a8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:35:44.782942 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ad272-8ecc-4cf7-86f5-a6c4b7583a8b service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:35:44.801813 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 667/1328] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:35:46.871523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd3a2e-3733-48ea-916e-dd46a39638ae None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:46.872121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd3a2e-3733-48ea-916e-dd46a39638ae None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:46.872121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd3a2e-3733-48ea-916e-dd46a39638ae None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:46.873506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 662/1329] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:35:46.879274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc4e92-b3a6-4af9-929f-8fa9a888166d None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:46.879717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc4e92-b3a6-4af9-929f-8fa9a888166d None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:46.879818 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc4e92-b3a6-4af9-929f-8fa9a888166d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:46.947985 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-edfc4e92-b3a6-4af9-929f-8fa9a888166d None None] MFA Rules not processed for user `c23f5930d01c4d6f819cc9308da0b14e`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:35:46.949193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-edfc4e92-b3a6-4af9-929f-8fa9a888166d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:47.017612 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:47.057080 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 668/1330] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:35:46 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:35:47.095482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd3a2e-3733-48ea-916e-dd46a39638ae None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:35:47.097022 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-75dd3a2e-3733-48ea-916e-dd46a39638ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:47.201854 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd3a2e-3733-48ea-916e-dd46a39638ae None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:35:47.222573 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:47.281348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b2888-c360-44d5-9cb2-7dfbc52c996a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:35:47.282090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b2888-c360-44d5-9cb2-7dfbc52c996a service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:47.282458 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b2888-c360-44d5-9cb2-7dfbc52c996a service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:47.282938 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b2888-c360-44d5-9cb2-7dfbc52c996a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:47.287149 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-909b2888-c360-44d5-9cb2-7dfbc52c996a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:47.412992 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b2888-c360-44d5-9cb2-7dfbc52c996a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:35:47.414512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b2888-c360-44d5-9cb2-7dfbc52c996a service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:35:47.434074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 663/1331] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:35:47 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:35:49.721140 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17836941-c9c8-474e-b4c5-0e1bc3f5ca34 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:49.721972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17836941-c9c8-474e-b4c5-0e1bc3f5ca34 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:49.721972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17836941-c9c8-474e-b4c5-0e1bc3f5ca34 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:49.724338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 669/1332] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:35:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:35:49.729973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bd4f5-d3ec-4f93-b44b-2718e6f647bb None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:49.730447 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bd4f5-d3ec-4f93-b44b-2718e6f647bb None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:49.730550 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bd4f5-d3ec-4f93-b44b-2718e6f647bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:49.797420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:49.800498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-d65bd4f5-d3ec-4f93-b44b-2718e6f647bb None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:35:49.802380 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d65bd4f5-d3ec-4f93-b44b-2718e6f647bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:49.904342 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 664/1333] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:35:49.950431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17836941-c9c8-474e-b4c5-0e1bc3f5ca34 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:35:49.952037 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-17836941-c9c8-474e-b4c5-0e1bc3f5ca34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:50.071772 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17836941-c9c8-474e-b4c5-0e1bc3f5ca34 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:35:50.150427 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197801cf-9170-43fe-89ba-d991203e0907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:35:50.151084 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197801cf-9170-43fe-89ba-d991203e0907 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:50.151373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197801cf-9170-43fe-89ba-d991203e0907 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:50.151657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197801cf-9170-43fe-89ba-d991203e0907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:50.153297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-197801cf-9170-43fe-89ba-d991203e0907 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:50.182207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:50.279707 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197801cf-9170-43fe-89ba-d991203e0907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:35:50.281167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197801cf-9170-43fe-89ba-d991203e0907 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:35:50.300210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 670/1334] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:35:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:35:52.393761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f321793-18a6-4d00-8afb-da626d9ef818 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:52.394688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f321793-18a6-4d00-8afb-da626d9ef818 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:52.394688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f321793-18a6-4d00-8afb-da626d9ef818 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:52.396895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 665/1335] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:35:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:35:52.405451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c308aac-6546-4a48-92c9-096063c65ffa None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:52.405867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c308aac-6546-4a48-92c9-096063c65ffa None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:52.406018 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c308aac-6546-4a48-92c9-096063c65ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:52.479889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-3c308aac-6546-4a48-92c9-096063c65ffa None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:35:52.481005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3c308aac-6546-4a48-92c9-096063c65ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:52.564201 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:52.599073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 671/1336] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:35:52.639704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f321793-18a6-4d00-8afb-da626d9ef818 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:35:52.641366 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-7f321793-18a6-4d00-8afb-da626d9ef818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:52.750737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f321793-18a6-4d00-8afb-da626d9ef818 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:35:52.777900 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:52.824984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b730bda-6f85-42e1-9c55-85f58af53307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:35:52.825667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b730bda-6f85-42e1-9c55-85f58af53307 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:52.825946 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b730bda-6f85-42e1-9c55-85f58af53307 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:52.826215 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b730bda-6f85-42e1-9c55-85f58af53307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:52.827878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6b730bda-6f85-42e1-9c55-85f58af53307 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:52.938506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b730bda-6f85-42e1-9c55-85f58af53307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:35:52.939990 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b730bda-6f85-42e1-9c55-85f58af53307 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:35:52.960830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 666/1337] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:35:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:35:55.349372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035edcb-aed2-4dfd-89b8-fbf74a6c654b None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:55.350103 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035edcb-aed2-4dfd-89b8-fbf74a6c654b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:55.350103 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035edcb-aed2-4dfd-89b8-fbf74a6c654b None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:55.352837 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 672/1338] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:35:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:35:55.363205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6883c03-6f36-45e7-83d7-288e25e2ff47 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:55.363920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6883c03-6f36-45e7-83d7-288e25e2ff47 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:55.364081 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6883c03-6f36-45e7-83d7-288e25e2ff47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:55.440867 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-f6883c03-6f36-45e7-83d7-288e25e2ff47 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:35:55.442082 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f6883c03-6f36-45e7-83d7-288e25e2ff47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:55.452394 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:55.547930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 667/1339] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:35:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:35:55.592826 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035edcb-aed2-4dfd-89b8-fbf74a6c654b None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:35:55.594967 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a035edcb-aed2-4dfd-89b8-fbf74a6c654b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:55.712979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035edcb-aed2-4dfd-89b8-fbf74a6c654b None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:35:55.800983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3cb923-2f0e-45f7-9da5-59711b074912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:35:55.801568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cb923-2f0e-45f7-9da5-59711b074912 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:55.801855 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cb923-2f0e-45f7-9da5-59711b074912 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:55.802128 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3cb923-2f0e-45f7-9da5-59711b074912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:55.803817 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4d3cb923-2f0e-45f7-9da5-59711b074912 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:35:55.840166 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:35:55.917051 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3cb923-2f0e-45f7-9da5-59711b074912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:35:55.918506 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3cb923-2f0e-45f7-9da5-59711b074912 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:35:55.939175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 673/1340] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:35:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:35:59.973054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0248bf7-fa2d-4cc2-be7e-3f9e8310ffd1 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:59.973624 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0248bf7-fa2d-4cc2-be7e-3f9e8310ffd1 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:59.973624 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0248bf7-fa2d-4cc2-be7e-3f9e8310ffd1 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:35:59.975085 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 668/1341] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:35:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:35:59.980565 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ce775-815e-4e23-8942-a0a7fda2557b None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:35:59.981024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ce775-815e-4e23-8942-a0a7fda2557b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:35:59.981024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ce775-815e-4e23-8942-a0a7fda2557b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:00.048248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-703ce775-815e-4e23-8942-a0a7fda2557b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:00.049466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-703ce775-815e-4e23-8942-a0a7fda2557b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:00.142252 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:00.155607 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 674/1342] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:00.195084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0248bf7-fa2d-4cc2-be7e-3f9e8310ffd1 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:00.196648 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b0248bf7-fa2d-4cc2-be7e-3f9e8310ffd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:00.304294 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0248bf7-fa2d-4cc2-be7e-3f9e8310ffd1 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:00.343822 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:00.380523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa5d99-70ba-45d2-b287-e807e986ae65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:00.381190 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa5d99-70ba-45d2-b287-e807e986ae65 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:00.381475 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa5d99-70ba-45d2-b287-e807e986ae65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:00.381749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa5d99-70ba-45d2-b287-e807e986ae65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:00.383417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-74aa5d99-70ba-45d2-b287-e807e986ae65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:00.502963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa5d99-70ba-45d2-b287-e807e986ae65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:00.504495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa5d99-70ba-45d2-b287-e807e986ae65 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:00.523273 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 669/1343] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:36:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:36:03.251878 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62e7d5-901c-4d32-b866-5587730271dd None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:03.252331 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62e7d5-901c-4d32-b866-5587730271dd None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:03.252438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62e7d5-901c-4d32-b866-5587730271dd None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:03.253850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 675/1344] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:03.259515 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6bcd1-5387-4169-9144-6e4881b089e4 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:03.259946 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6bcd1-5387-4169-9144-6e4881b089e4 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:03.260057 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c6bcd1-5387-4169-9144-6e4881b089e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:03.331679 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-78c6bcd1-5387-4169-9144-6e4881b089e4 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:03.332890 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-78c6bcd1-5387-4169-9144-6e4881b089e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:03.352782 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:03.440540 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 670/1345] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:36:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:03.482535 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62e7d5-901c-4d32-b866-5587730271dd None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:03.484341 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9b62e7d5-901c-4d32-b866-5587730271dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:03.589879 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62e7d5-901c-4d32-b866-5587730271dd None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:03.665652 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a1e1d-5293-4826-ab48-687d2ec3f448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:03.666312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a1e1d-5293-4826-ab48-687d2ec3f448 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:03.666560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a1e1d-5293-4826-ab48-687d2ec3f448 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:03.666854 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a1e1d-5293-4826-ab48-687d2ec3f448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:03.668507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8a7a1e1d-5293-4826-ab48-687d2ec3f448 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:03.715986 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:03.786583 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a1e1d-5293-4826-ab48-687d2ec3f448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:03.788082 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a1e1d-5293-4826-ab48-687d2ec3f448 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:03.807142 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 676/1346] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:36:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:36:07.785912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06065107-bf86-4b36-9ca8-d7a262b983fd None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:07.786466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06065107-bf86-4b36-9ca8-d7a262b983fd None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:07.786466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06065107-bf86-4b36-9ca8-d7a262b983fd None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:07.787885 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 671/1347] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:36:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:07.793571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83664fa1-76c8-47ab-92df-a871c9b2402a None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:07.794041 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83664fa1-76c8-47ab-92df-a871c9b2402a None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:07.794218 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83664fa1-76c8-47ab-92df-a871c9b2402a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:07.859706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-83664fa1-76c8-47ab-92df-a871c9b2402a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:07.860981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-83664fa1-76c8-47ab-92df-a871c9b2402a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:07.980983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:07.987070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 677/1348] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:36:08.022265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06065107-bf86-4b36-9ca8-d7a262b983fd None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:08.024022 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-06065107-bf86-4b36-9ca8-d7a262b983fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:08.131695 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06065107-bf86-4b36-9ca8-d7a262b983fd None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:08.184991 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:08.213381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a85485-7cbb-4eb1-95b5-b59293cbfc8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:08.214043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a85485-7cbb-4eb1-95b5-b59293cbfc8b service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:08.214324 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a85485-7cbb-4eb1-95b5-b59293cbfc8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:08.214597 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a85485-7cbb-4eb1-95b5-b59293cbfc8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:08.216299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d4a85485-7cbb-4eb1-95b5-b59293cbfc8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:08.344558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a85485-7cbb-4eb1-95b5-b59293cbfc8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:08.346105 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a85485-7cbb-4eb1-95b5-b59293cbfc8b service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:08.367400 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 672/1349] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:36:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:36:13.293385 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c250dad-efcd-441e-9ffd-d75fac01022c None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:13.293939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c250dad-efcd-441e-9ffd-d75fac01022c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:13.294290 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c250dad-efcd-441e-9ffd-d75fac01022c None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:13.296078 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 678/1350] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jul 21 05:36:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:13.302389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c4589-c16a-4ddf-a06a-258beb0cc788 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:13.302986 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c4589-c16a-4ddf-a06a-258beb0cc788 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:13.303383 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c4589-c16a-4ddf-a06a-258beb0cc788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:13.425059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-db8c4589-c16a-4ddf-a06a-258beb0cc788 None None] MFA Rules not processed for user `03d868598e39448881b43d299253d821`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:13.426270 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-db8c4589-c16a-4ddf-a06a-258beb0cc788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:13.445746 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:13.560108 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 673/1351] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jul 21 05:36:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:17.743907 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524d4d8-7845-46e0-8559-1102d3b1e1c3 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:17.744517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524d4d8-7845-46e0-8559-1102d3b1e1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:17.744517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524d4d8-7845-46e0-8559-1102d3b1e1c3 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:17.746116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 679/1352] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jul 21 05:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:17.751144 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513a1ea-ba9b-4a56-89cc-c63099e0accf None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:17.751617 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513a1ea-ba9b-4a56-89cc-c63099e0accf None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:17.751964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513a1ea-ba9b-4a56-89cc-c63099e0accf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:17.833836 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-1513a1ea-ba9b-4a56-89cc-c63099e0accf None None] MFA Rules not processed for user `03d868598e39448881b43d299253d821`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:17.835110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1513a1ea-ba9b-4a56-89cc-c63099e0accf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:17.913598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:17.957430 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 674/1353] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jul 21 05:36:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:19.132510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e1587-9534-4162-a07e-1a9c61918e1b None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:19.133049 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e1587-9534-4162-a07e-1a9c61918e1b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:19.133150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9e1587-9534-4162-a07e-1a9c61918e1b None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:19.134538 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 680/1354] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jul 21 05:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:19.140062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099fab4-f356-4a82-9da6-bfb0bb0f4ae8 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:19.140542 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099fab4-f356-4a82-9da6-bfb0bb0f4ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:19.140744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5099fab4-f356-4a82-9da6-bfb0bb0f4ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:19.217397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-5099fab4-f356-4a82-9da6-bfb0bb0f4ae8 None None] MFA Rules not processed for user `03d868598e39448881b43d299253d821`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:19.218639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5099fab4-f356-4a82-9da6-bfb0bb0f4ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:19.324447 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 675/1355] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jul 21 05:36:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:22.729350 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663d845-9730-4613-9430-c05e2a0bfcdc None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:22.729889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663d845-9730-4613-9430-c05e2a0bfcdc None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:22.729889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663d845-9730-4613-9430-c05e2a0bfcdc None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:22.731273 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 681/1356] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:22.736969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9dbd3-876d-4f4c-b929-c57c4151c3d4 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:22.737454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9dbd3-876d-4f4c-b929-c57c4151c3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:22.737722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9dbd3-876d-4f4c-b929-c57c4151c3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:22.786722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:22.824846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-25e9dbd3-876d-4f4c-b929-c57c4151c3d4 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:22.826212 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-25e9dbd3-876d-4f4c-b929-c57c4151c3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:22.931065 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 676/1357] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:22.982809 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf79e8-7bb5-45d9-9fe7-a51d332774ca None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:22.983143 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf79e8-7bb5-45d9-9fe7-a51d332774ca None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:22.983412 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf79e8-7bb5-45d9-9fe7-a51d332774ca None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:22.984868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 682/1358] 10.222.0.44 () {54 vars in 962 bytes} [Thu Jul 21 05:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:22.994428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c71934-06e0-4d2d-83ce-cb92cd91d8c8 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:22.994836 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c71934-06e0-4d2d-83ce-cb92cd91d8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:22.995245 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c71934-06e0-4d2d-83ce-cb92cd91d8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:23.073771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-29c71934-06e0-4d2d-83ce-cb92cd91d8c8 None None] MFA Rules not processed for user `928d014648cd41d7b58e7e01845eb748`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:23.074958 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-29c71934-06e0-4d2d-83ce-cb92cd91d8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:23.125980 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:23.198497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 677/1359] 10.222.0.44 () {58 vars in 1059 bytes} [Thu Jul 21 05:36:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:23.206184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf79e8-7bb5-45d9-9fe7-a51d332774ca None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:23.207704 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-c2cf79e8-7bb5-45d9-9fe7-a51d332774ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:23.354437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf79e8-7bb5-45d9-9fe7-a51d332774ca None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:23.429906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:23.473582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af4475-686d-4cfd-83b2-91e15d2b9568 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:23.474329 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af4475-686d-4cfd-83b2-91e15d2b9568 service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:23.474707 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af4475-686d-4cfd-83b2-91e15d2b9568 service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:23.475123 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af4475-686d-4cfd-83b2-91e15d2b9568 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:23.476822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d1af4475-686d-4cfd-83b2-91e15d2b9568 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:23.572487 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af4475-686d-4cfd-83b2-91e15d2b9568 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:23.574260 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af4475-686d-4cfd-83b2-91e15d2b9568 service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:23.593815 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 683/1360] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:36:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:36:25.845937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16039e09-b148-46f0-b6ea-9f53a2afe1c7 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:25.846392 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16039e09-b148-46f0-b6ea-9f53a2afe1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:25.846575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16039e09-b148-46f0-b6ea-9f53a2afe1c7 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:25.848139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 678/1361] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:36:25.854256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af9a04-2d06-4ad1-b0bf-f894a7fd6b88 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:25.854730 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af9a04-2d06-4ad1-b0bf-f894a7fd6b88 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:25.855144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af9a04-2d06-4ad1-b0bf-f894a7fd6b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:25.927412 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-00af9a04-2d06-4ad1-b0bf-f894a7fd6b88 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:25.928637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-00af9a04-2d06-4ad1-b0bf-f894a7fd6b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:25.974159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:26.037318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 684/1362] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:26.085280 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16039e09-b148-46f0-b6ea-9f53a2afe1c7 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:26.088169 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-16039e09-b148-46f0-b6ea-9f53a2afe1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:26.276576 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16039e09-b148-46f0-b6ea-9f53a2afe1c7 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:26.287960 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:26.429671 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9025e964-c1c5-4554-92d7-503fdcffc0d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:26.430363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025e964-c1c5-4554-92d7-503fdcffc0d8 service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:26.430716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025e964-c1c5-4554-92d7-503fdcffc0d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:26.431090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9025e964-c1c5-4554-92d7-503fdcffc0d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:26.433027 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-9025e964-c1c5-4554-92d7-503fdcffc0d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:26.527798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025e964-c1c5-4554-92d7-503fdcffc0d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:26.529431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9025e964-c1c5-4554-92d7-503fdcffc0d8 service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:26.548789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 679/1363] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:36:28.910424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0d6b5-4acc-4dab-b371-6c311fe76034 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:28.910961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0d6b5-4acc-4dab-b371-6c311fe76034 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:28.910961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0d6b5-4acc-4dab-b371-6c311fe76034 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:28.912590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 685/1364] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:36:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:28.917709 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202730a0-0281-4c93-bfc0-65f18c86d002 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:28.918156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202730a0-0281-4c93-bfc0-65f18c86d002 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:28.918252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202730a0-0281-4c93-bfc0-65f18c86d002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:28.944041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:28.997345 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-202730a0-0281-4c93-bfc0-65f18c86d002 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:28.998538 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-202730a0-0281-4c93-bfc0-65f18c86d002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:29.111058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 680/1365] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:29.152777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08dc6e-e3ea-4a2c-940d-12992020f71d None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:29.153172 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08dc6e-e3ea-4a2c-940d-12992020f71d None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:29.153711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08dc6e-e3ea-4a2c-940d-12992020f71d None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:29.155382 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 686/1366] 10.222.0.44 () {54 vars in 962 bytes} [Thu Jul 21 05:36:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:29.161564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113dddb-738b-4853-bdb3-9be390f4137f None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:29.161850 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113dddb-738b-4853-bdb3-9be390f4137f None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:29.162224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113dddb-738b-4853-bdb3-9be390f4137f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:29.241889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-2113dddb-738b-4853-bdb3-9be390f4137f None None] MFA Rules not processed for user `928d014648cd41d7b58e7e01845eb748`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:29.242981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2113dddb-738b-4853-bdb3-9be390f4137f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:29.279740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:29.365630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 681/1367] 10.222.0.44 () {58 vars in 1059 bytes} [Thu Jul 21 05:36:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:29.373482 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08dc6e-e3ea-4a2c-940d-12992020f71d None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:29.375147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ab08dc6e-e3ea-4a2c-940d-12992020f71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:29.551751 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08dc6e-e3ea-4a2c-940d-12992020f71d None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:29.572858 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:29.676201 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed2d6d-0da4-4a80-90a1-106f1206d96c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:29.676862 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed2d6d-0da4-4a80-90a1-106f1206d96c service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:29.677136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed2d6d-0da4-4a80-90a1-106f1206d96c service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:29.677409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed2d6d-0da4-4a80-90a1-106f1206d96c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:29.679039 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d8ed2d6d-0da4-4a80-90a1-106f1206d96c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:29.776314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed2d6d-0da4-4a80-90a1-106f1206d96c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:29.778136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed2d6d-0da4-4a80-90a1-106f1206d96c service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:29.803291 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 687/1368] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:36:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:36:30.830088 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c9369-8d51-4241-ae09-bf216e279f9e None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:30.830629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c9369-8d51-4241-ae09-bf216e279f9e None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:30.830629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c9369-8d51-4241-ae09-bf216e279f9e None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:30.833210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 682/1369] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:30.838793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bdd210-cde4-4ee9-9f14-8ccdcec3c649 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:30.839319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bdd210-cde4-4ee9-9f14-8ccdcec3c649 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:30.839729 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bdd210-cde4-4ee9-9f14-8ccdcec3c649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:30.871729 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:30.909574 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-c6bdd210-cde4-4ee9-9f14-8ccdcec3c649 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:30.911323 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-c6bdd210-cde4-4ee9-9f14-8ccdcec3c649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:31.017638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 688/1370] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:31.354987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e537e7e-197f-49e7-93f6-664c29f872ff None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:31.354987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e537e7e-197f-49e7-93f6-664c29f872ff None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:31.354987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e537e7e-197f-49e7-93f6-664c29f872ff None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:31.357385 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 683/1371] 10.222.0.44 () {54 vars in 964 bytes} [Thu Jul 21 05:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:31.362450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba295f6d-f2fb-467e-a430-17d339c08229 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:31.362450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba295f6d-f2fb-467e-a430-17d339c08229 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:31.362450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba295f6d-f2fb-467e-a430-17d339c08229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:31.439662 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-ba295f6d-f2fb-467e-a430-17d339c08229 None None] MFA Rules not processed for user `f3f76371c6a04fc5a477d6ef1908a08e`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:31.440867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ba295f6d-f2fb-467e-a430-17d339c08229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:31.489730 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:31.521562 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 689/1372] 10.222.0.44 () {58 vars in 1061 bytes} [Thu Jul 21 05:36:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:36:31.529078 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e537e7e-197f-49e7-93f6-664c29f872ff None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:31.530575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5e537e7e-197f-49e7-93f6-664c29f872ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:31.643271 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e537e7e-197f-49e7-93f6-664c29f872ff None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:31.695512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:31.732987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3992b67-a034-4d6c-a33a-290a562a4c64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:31.733861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3992b67-a034-4d6c-a33a-290a562a4c64 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:31.734234 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3992b67-a034-4d6c-a33a-290a562a4c64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:31.734614 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3992b67-a034-4d6c-a33a-290a562a4c64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:31.736412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c3992b67-a034-4d6c-a33a-290a562a4c64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:31.864009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3992b67-a034-4d6c-a33a-290a562a4c64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:31.865845 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3992b67-a034-4d6c-a33a-290a562a4c64 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:31.891197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 684/1373] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:36:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:36:31.903582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddfdb4-5d04-499d-8ae3-084d22f58bd5 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:31.903582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddfdb4-5d04-499d-8ae3-084d22f58bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:31.903582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddfdb4-5d04-499d-8ae3-084d22f58bd5 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:31.906025 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 690/1374] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:36:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:31.911416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d9bf1-1f35-4602-9126-0bbca3575903 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:31.911622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d9bf1-1f35-4602-9126-0bbca3575903 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:31.912459 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d9bf1-1f35-4602-9126-0bbca3575903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:31.977500 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-e57d9bf1-1f35-4602-9126-0bbca3575903 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:31.978590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e57d9bf1-1f35-4602-9126-0bbca3575903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:32.000764 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:32.087341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 685/1375] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:32.121084 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebddfdb4-5d04-499d-8ae3-084d22f58bd5 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:32.123292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ebddfdb4-5d04-499d-8ae3-084d22f58bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:32.282344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebddfdb4-5d04-499d-8ae3-084d22f58bd5 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:32.335419 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:32.396112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac742c-ede2-40c8-8572-0275018e1368 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:32.396763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac742c-ede2-40c8-8572-0275018e1368 service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:32.397044 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac742c-ede2-40c8-8572-0275018e1368 service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:32.397315 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac742c-ede2-40c8-8572-0275018e1368 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:32.398927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-6dac742c-ede2-40c8-8572-0275018e1368 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:32.489104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac742c-ede2-40c8-8572-0275018e1368 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:32.492040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac742c-ede2-40c8-8572-0275018e1368 service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:32.510139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 691/1376] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:36:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:36:34.605788 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef51f9a-b1f8-4779-8686-6ef0ccf235a3 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:34.606313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef51f9a-b1f8-4779-8686-6ef0ccf235a3 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:34.606313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef51f9a-b1f8-4779-8686-6ef0ccf235a3 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:34.607656 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 686/1377] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:34.613227 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8f707-f360-415c-b1e6-3a08c10772c2 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:34.613693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8f707-f360-415c-b1e6-3a08c10772c2 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:34.613797 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8f707-f360-415c-b1e6-3a08c10772c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:34.690693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-d2c8f707-f360-415c-b1e6-3a08c10772c2 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:34.692137 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d2c8f707-f360-415c-b1e6-3a08c10772c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:34.701356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:34.796786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 692/1378] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:34.847809 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef51f9a-b1f8-4779-8686-6ef0ccf235a3 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:34.849600 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-eef51f9a-b1f8-4779-8686-6ef0ccf235a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:35.004551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef51f9a-b1f8-4779-8686-6ef0ccf235a3 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:35.008573 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:35.123890 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93e08b-ba08-499c-b811-2c89a19a9484 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:35.124553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93e08b-ba08-499c-b811-2c89a19a9484 service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:35.124838 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93e08b-ba08-499c-b811-2c89a19a9484 service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:35.125111 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93e08b-ba08-499c-b811-2c89a19a9484 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:35.126771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4b93e08b-ba08-499c-b811-2c89a19a9484 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:35.224988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93e08b-ba08-499c-b811-2c89a19a9484 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:35.226636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b93e08b-ba08-499c-b811-2c89a19a9484 service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:35.245080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 687/1379] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:36:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:36:37.315802 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8ccfa-ef61-4f25-bb78-6c55517b1383 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:37.316359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8ccfa-ef61-4f25-bb78-6c55517b1383 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:37.316359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd8ccfa-ef61-4f25-bb78-6c55517b1383 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:37.317887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 693/1380] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:37.323837 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb36169-2b61-4adb-9fa7-604193b56d71 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:37.324432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb36169-2b61-4adb-9fa7-604193b56d71 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:37.324432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb36169-2b61-4adb-9fa7-604193b56d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:37.336271 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:37.407843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-3fb36169-2b61-4adb-9fa7-604193b56d71 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:37.409493 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-3fb36169-2b61-4adb-9fa7-604193b56d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:37.518628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 688/1381] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:37.564489 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8ccfa-ef61-4f25-bb78-6c55517b1383 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:37.566051 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0cd8ccfa-ef61-4f25-bb78-6c55517b1383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:37.718735 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:37.730144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd8ccfa-ef61-4f25-bb78-6c55517b1383 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:37.846937 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b835c-b88d-46b2-9d3c-c0ab4960260e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:37.847538 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b835c-b88d-46b2-9d3c-c0ab4960260e service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:37.847821 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b835c-b88d-46b2-9d3c-c0ab4960260e service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:37.848109 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b835c-b88d-46b2-9d3c-c0ab4960260e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:37.849718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-551b835c-b88d-46b2-9d3c-c0ab4960260e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:37.942875 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b835c-b88d-46b2-9d3c-c0ab4960260e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:37.944492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b835c-b88d-46b2-9d3c-c0ab4960260e service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:37.955419 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:37.964687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 694/1382] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:36:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:36:40.190409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd2bcd-7916-4b6d-b6e3-089419ad43ba None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:40.191085 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd2bcd-7916-4b6d-b6e3-089419ad43ba None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:40.191085 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd2bcd-7916-4b6d-b6e3-089419ad43ba None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:40.192532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 689/1383] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:40.198650 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6d3bc-576b-418b-a943-385f4b8c302e None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:40.199170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6d3bc-576b-418b-a943-385f4b8c302e None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:40.199277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6d3bc-576b-418b-a943-385f4b8c302e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:40.285699 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-ccf6d3bc-576b-418b-a943-385f4b8c302e None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:40.287390 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ccf6d3bc-576b-418b-a943-385f4b8c302e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:40.414491 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 695/1384] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:40.471521 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd2bcd-7916-4b6d-b6e3-089419ad43ba None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:40.473823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-42dd2bcd-7916-4b6d-b6e3-089419ad43ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:40.578209 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:40.652994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd2bcd-7916-4b6d-b6e3-089419ad43ba None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:40.779980 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693bafd-70bd-4c56-b4e6-04daaa94fcd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:40.780626 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693bafd-70bd-4c56-b4e6-04daaa94fcd1 service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:40.780933 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693bafd-70bd-4c56-b4e6-04daaa94fcd1 service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:40.781168 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693bafd-70bd-4c56-b4e6-04daaa94fcd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:40.782891 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2693bafd-70bd-4c56-b4e6-04daaa94fcd1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:40.838044 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:40.882718 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693bafd-70bd-4c56-b4e6-04daaa94fcd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:40.884429 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693bafd-70bd-4c56-b4e6-04daaa94fcd1 service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:40.908019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 690/1385] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:36:43.082205 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62cdac-90f9-4793-99bb-0fc01b58aa09 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:43.082974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62cdac-90f9-4793-99bb-0fc01b58aa09 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:43.082974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62cdac-90f9-4793-99bb-0fc01b58aa09 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:43.085088 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 696/1386] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:36:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:43.091175 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba97bae-41a9-451e-b1b5-395910d465a4 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:43.091630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba97bae-41a9-451e-b1b5-395910d465a4 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:43.091903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba97bae-41a9-451e-b1b5-395910d465a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:43.175569 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-5ba97bae-41a9-451e-b1b5-395910d465a4 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:43.177226 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5ba97bae-41a9-451e-b1b5-395910d465a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:43.292406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 691/1387] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:43.342822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62cdac-90f9-4793-99bb-0fc01b58aa09 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:43.344904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ee62cdac-90f9-4793-99bb-0fc01b58aa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:43.439936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:43.521764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62cdac-90f9-4793-99bb-0fc01b58aa09 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:43.646711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4af4a-c83b-4810-ba74-ab0478c88b92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:43.647346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4af4a-c83b-4810-ba74-ab0478c88b92 service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:43.647639 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4af4a-c83b-4810-ba74-ab0478c88b92 service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:43.648019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4af4a-c83b-4810-ba74-ab0478c88b92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:43.649791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-83c4af4a-c83b-4810-ba74-ab0478c88b92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:43.694789 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:43.752149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4af4a-c83b-4810-ba74-ab0478c88b92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:43.753984 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4af4a-c83b-4810-ba74-ab0478c88b92 service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:43.773761 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 697/1388] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:36:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:36:46.091303 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22d42a-5305-49ba-85e0-e0145e879fa3 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:46.091842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22d42a-5305-49ba-85e0-e0145e879fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:46.091842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a22d42a-5305-49ba-85e0-e0145e879fa3 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:46.093235 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 692/1389] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:46.098660 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cdff1-3c7e-4d41-a178-bb21111a845e None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:46.099127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cdff1-3c7e-4d41-a178-bb21111a845e None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:46.099226 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cdff1-3c7e-4d41-a178-bb21111a845e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:46.173176 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-de8cdff1-3c7e-4d41-a178-bb21111a845e None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:46.174382 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-de8cdff1-3c7e-4d41-a178-bb21111a845e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:46.282285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 698/1390] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:46.326729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22d42a-5305-49ba-85e0-e0145e879fa3 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:46.328367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4a22d42a-5305-49ba-85e0-e0145e879fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:46.360678 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:46.479049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a22d42a-5305-49ba-85e0-e0145e879fa3 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:46.588408 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:46.601144 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27f193-1ce5-420a-9ab4-12aef465afc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:46.601805 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27f193-1ce5-420a-9ab4-12aef465afc0 service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:46.602087 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27f193-1ce5-420a-9ab4-12aef465afc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:46.602359 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27f193-1ce5-420a-9ab4-12aef465afc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:46.604036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ae27f193-1ce5-420a-9ab4-12aef465afc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:46.701769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27f193-1ce5-420a-9ab4-12aef465afc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:46.703441 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27f193-1ce5-420a-9ab4-12aef465afc0 service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:46.722630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 693/1391] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:36:46.830221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d094f-de6b-4d7d-a1f3-210909140c70 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:46.830500 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d094f-de6b-4d7d-a1f3-210909140c70 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:46.830773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d094f-de6b-4d7d-a1f3-210909140c70 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:46.832213 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 699/1392] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:46.838183 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379318cd-cf28-4d69-bbf2-7712da9e7e8d None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:46.838549 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379318cd-cf28-4d69-bbf2-7712da9e7e8d None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:46.838828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379318cd-cf28-4d69-bbf2-7712da9e7e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:46.914364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-379318cd-cf28-4d69-bbf2-7712da9e7e8d None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:46.915556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-379318cd-cf28-4d69-bbf2-7712da9e7e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:46.979396 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:47.019562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 694/1393] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:47.075533 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84605dd0-971f-4ead-aaf6-db0364814aad None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:47.075833 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84605dd0-971f-4ead-aaf6-db0364814aad None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:47.076071 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84605dd0-971f-4ead-aaf6-db0364814aad None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:47.077635 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 700/1394] 10.222.0.44 () {54 vars in 966 bytes} [Thu Jul 21 05:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:47.082775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcde41d-e012-4aed-8928-1820dfd843e1 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:47.083086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcde41d-e012-4aed-8928-1820dfd843e1 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:47.083398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcde41d-e012-4aed-8928-1820dfd843e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:47.163652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-5dcde41d-e012-4aed-8928-1820dfd843e1 None None] MFA Rules not processed for user `2707c6967a0a419a98ae142ff1d1a961`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:47.164944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5dcde41d-e012-4aed-8928-1820dfd843e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:47.243689 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 695/1395] 10.222.0.44 () {58 vars in 1063 bytes} [Thu Jul 21 05:36:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:47.251711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84605dd0-971f-4ead-aaf6-db0364814aad None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:47.253278 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-84605dd0-971f-4ead-aaf6-db0364814aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:47.268097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:47.359864 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84605dd0-971f-4ead-aaf6-db0364814aad None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:47.433317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dfd49-dd30-41eb-a4f0-9b90026f7306 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2707c6967a0a419a98ae142ff1d1a961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:47.433937 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dfd49-dd30-41eb-a4f0-9b90026f7306 service glance] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:47.434239 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dfd49-dd30-41eb-a4f0-9b90026f7306 service glance] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:47.434513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dfd49-dd30-41eb-a4f0-9b90026f7306 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:47.436186 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5e1dfd49-dd30-41eb-a4f0-9b90026f7306 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:47.508405 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:47.551850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1dfd49-dd30-41eb-a4f0-9b90026f7306 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:47.553302 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1dfd49-dd30-41eb-a4f0-9b90026f7306 service glance] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:47.571517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 701/1396] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:36:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:36:48.287757 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e1dcb-ba00-4516-9db6-c9b0e36b22c4 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:48.288219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e1dcb-ba00-4516-9db6-c9b0e36b22c4 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:48.288219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e1dcb-ba00-4516-9db6-c9b0e36b22c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:48.353308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-030e1dcb-ba00-4516-9db6-c9b0e36b22c4 None None] MFA Rules not processed for user `4d04998b636649d98103b1cd5f3a0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:48.354507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-030e1dcb-ba00-4516-9db6-c9b0e36b22c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:48.362986 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:48.444316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 696/1397] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jul 21 05:36:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:48.491424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3487a-db44-4ef3-9d24-26a824adeafa None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:48.491909 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3487a-db44-4ef3-9d24-26a824adeafa None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:48.492099 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3487a-db44-4ef3-9d24-26a824adeafa None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:48.493525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 702/1398] 10.222.0.44 () {54 vars in 943 bytes} [Thu Jul 21 05:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:48.499878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd9107-85c1-4bfc-af40-66f42f13ec2b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:48.500282 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd9107-85c1-4bfc-af40-66f42f13ec2b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:48.500564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd9107-85c1-4bfc-af40-66f42f13ec2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:48.590838 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-cdfd9107-85c1-4bfc-af40-66f42f13ec2b None None] MFA Rules not processed for user `61b61188b5b94a51ba1ae1a88b78b89f`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:48.592036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-cdfd9107-85c1-4bfc-af40-66f42f13ec2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:48.685283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:48.708630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 697/1399] 10.222.0.44 () {58 vars in 1040 bytes} [Thu Jul 21 05:36:48 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:36:48.718235 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3487a-db44-4ef3-9d24-26a824adeafa None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:48.719805 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-07d3487a-db44-4ef3-9d24-26a824adeafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:48.791181 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848d33f-a6e4-4b41-b340-9f5cc68ce5a5 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:48.791453 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848d33f-a6e4-4b41-b340-9f5cc68ce5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:48.791748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848d33f-a6e4-4b41-b340-9f5cc68ce5a5 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:48.793161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 698/1400] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:36:48.798243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29808503-860f-4f6b-8a83-35d9287add7a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:48.798530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29808503-860f-4f6b-8a83-35d9287add7a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:48.798843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29808503-860f-4f6b-8a83-35d9287add7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:48.866896 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3487a-db44-4ef3-9d24-26a824adeafa None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:48.867683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-29808503-860f-4f6b-8a83-35d9287add7a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:48.868906 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-29808503-860f-4f6b-8a83-35d9287add7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:48.904997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:48.967082 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 699/1401] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:36:48.974315 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbbda6-adc7-498c-ad5e-71a4fee2ce3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61b61188b5b94a51ba1ae1a88b78b89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:48.974975 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbbda6-adc7-498c-ad5e-71a4fee2ce3f service swift] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:48.975284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbbda6-adc7-498c-ad5e-71a4fee2ce3f service swift] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:48.975609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbbda6-adc7-498c-ad5e-71a4fee2ce3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:48.977278 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b5fbbda6-adc7-498c-ad5e-71a4fee2ce3f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:49.015559 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29808503-860f-4f6b-8a83-35d9287add7a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:49.016838 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-29808503-860f-4f6b-8a83-35d9287add7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:49.063179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fbbda6-adc7-498c-ad5e-71a4fee2ce3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:49.064798 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fbbda6-adc7-498c-ad5e-71a4fee2ce3f service swift] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:49.073310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 703/1402] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jul 21 05:36:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 05:36:49.083040 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:49.137551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29808503-860f-4f6b-8a83-35d9287add7a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:49.266487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f860e-b0c2-4b17-8efe-578800c80877 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:49.267185 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f860e-b0c2-4b17-8efe-578800c80877 service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:49.267476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f860e-b0c2-4b17-8efe-578800c80877 service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:49.267860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f860e-b0c2-4b17-8efe-578800c80877 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:49.269583 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-166f860e-b0c2-4b17-8efe-578800c80877 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:49.337543 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:49.371973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f860e-b0c2-4b17-8efe-578800c80877 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:49.373986 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f860e-b0c2-4b17-8efe-578800c80877 service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:49.397223 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 700/1403] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:36:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:36:51.453545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3171ac8-6e6d-41d7-b2b1-c2149db9a4c2 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:51.454215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3171ac8-6e6d-41d7-b2b1-c2149db9a4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:51.454215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3171ac8-6e6d-41d7-b2b1-c2149db9a4c2 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:51.455527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 704/1404] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:51.460969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111f7bf-0158-4e9a-8c6f-852f03adacb8 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:51.461455 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111f7bf-0158-4e9a-8c6f-852f03adacb8 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:51.461545 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111f7bf-0158-4e9a-8c6f-852f03adacb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:51.544437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-4111f7bf-0158-4e9a-8c6f-852f03adacb8 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:51.545845 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4111f7bf-0158-4e9a-8c6f-852f03adacb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:51.684202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 701/1405] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:36:51.727990 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3171ac8-6e6d-41d7-b2b1-c2149db9a4c2 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:51.729882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d3171ac8-6e6d-41d7-b2b1-c2149db9a4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:51.821542 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:51.907692 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3171ac8-6e6d-41d7-b2b1-c2149db9a4c2 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:52.027433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8050b6-bbb4-491e-a3a4-a04d5bbc4a25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:52.028378 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8050b6-bbb4-491e-a3a4-a04d5bbc4a25 service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:52.028662 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8050b6-bbb4-491e-a3a4-a04d5bbc4a25 service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:52.028934 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8050b6-bbb4-491e-a3a4-a04d5bbc4a25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:52.030646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-bb8050b6-bbb4-491e-a3a4-a04d5bbc4a25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:52.065995 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:52.129785 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8050b6-bbb4-491e-a3a4-a04d5bbc4a25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:52.131754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8050b6-bbb4-491e-a3a4-a04d5bbc4a25 service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:52.151976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 705/1406] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:36:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:36:54.251147 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6d54d-2c8f-4fb7-84ec-b7826a20f935 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:54.251602 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6d54d-2c8f-4fb7-84ec-b7826a20f935 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:54.251712 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6d54d-2c8f-4fb7-84ec-b7826a20f935 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:54.259284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc51772-d39b-4edc-b041-9e0050c32fc4 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:54.259284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc51772-d39b-4edc-b041-9e0050c32fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:54.259284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc51772-d39b-4edc-b041-9e0050c32fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:54.260187 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 702/1407] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:36:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:36:54.328846 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-ddc51772-d39b-4edc-b041-9e0050c32fc4 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:36:54.330465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ddc51772-d39b-4edc-b041-9e0050c32fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:54.435130 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 706/1408] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:36:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:36:54.479562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6d54d-2c8f-4fb7-84ec-b7826a20f935 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:36:54.481187 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-84d6d54d-2c8f-4fb7-84ec-b7826a20f935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:54.523568 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:54.638575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6d54d-2c8f-4fb7-84ec-b7826a20f935 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:36:54.766106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:36:54.773057 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef660c3-7f02-4bae-9dd0-e8eac892fc16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:36:54.775166 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef660c3-7f02-4bae-9dd0-e8eac892fc16 service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:36:54.775166 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef660c3-7f02-4bae-9dd0-e8eac892fc16 service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:36:54.775166 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef660c3-7f02-4bae-9dd0-e8eac892fc16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:36:54.776113 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6ef660c3-7f02-4bae-9dd0-e8eac892fc16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:36:54.877692 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef660c3-7f02-4bae-9dd0-e8eac892fc16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:36:54.879906 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef660c3-7f02-4bae-9dd0-e8eac892fc16 service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:36:54.898790 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 703/1409] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:36:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:04.432895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:04.896535 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=47 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:11.138744 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cf008-f2a8-4308-a2ee-dbc02c4211ca None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:11.139466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cf008-f2a8-4308-a2ee-dbc02c4211ca None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:11.139466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cf008-f2a8-4308-a2ee-dbc02c4211ca None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:11.140854 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 707/1410] 10.222.0.44 () {54 vars in 918 bytes} [Thu Jul 21 05:37:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:11.146050 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5afcf-460e-4f92-99c4-09eaceca9f94 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:11.146498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5afcf-460e-4f92-99c4-09eaceca9f94 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:11.146605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c5afcf-460e-4f92-99c4-09eaceca9f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:11.238572 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-d2c5afcf-460e-4f92-99c4-09eaceca9f94 None None] MFA Rules not processed for user `03d868598e39448881b43d299253d821`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:11.239880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d2c5afcf-460e-4f92-99c4-09eaceca9f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:11.353471 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 704/1411] 10.222.0.44 () {58 vars in 1015 bytes} [Thu Jul 21 05:37:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:11.374948 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b78d0-afff-43d0-8f6a-c90d86082590 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:11.375268 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b78d0-afff-43d0-8f6a-c90d86082590 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:11.375539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b78d0-afff-43d0-8f6a-c90d86082590 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:11.376930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 708/1412] 10.222.0.44 () {54 vars in 918 bytes} [Thu Jul 21 05:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:11.381737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61e328-a5cd-4af8-8d11-ecfb4461bb24 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:11.382024 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61e328-a5cd-4af8-8d11-ecfb4461bb24 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:11.382313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61e328-a5cd-4af8-8d11-ecfb4461bb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:11.454636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:11.457742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-ce61e328-a5cd-4af8-8d11-ecfb4461bb24 None None] MFA Rules not processed for user `928d014648cd41d7b58e7e01845eb748`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:11.459721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ce61e328-a5cd-4af8-8d11-ecfb4461bb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:11.580491 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 705/1413] 10.222.0.44 () {58 vars in 1015 bytes} [Thu Jul 21 05:37:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:37:14.074405 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b892c-b2db-461d-ba4f-b2d3565e9c90 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:14.075325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b892c-b2db-461d-ba4f-b2d3565e9c90 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:14.075325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b892c-b2db-461d-ba4f-b2d3565e9c90 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:14.077456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 05:37:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:14.084079 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcd81c-24bc-4de1-ac5b-2a9e192f51b6 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:14.084522 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcd81c-24bc-4de1-ac5b-2a9e192f51b6 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:14.084658 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcd81c-24bc-4de1-ac5b-2a9e192f51b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:14.161312 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-c0dcd81c-24bc-4de1-ac5b-2a9e192f51b6 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:14.162626 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c0dcd81c-24bc-4de1-ac5b-2a9e192f51b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:14.223069 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:14.266855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 05:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:14.310380 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b892c-b2db-461d-ba4f-b2d3565e9c90 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:14.312423 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9a9b892c-b2db-461d-ba4f-b2d3565e9c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:14.469525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b892c-b2db-461d-ba4f-b2d3565e9c90 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:14.545081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:14.585746 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7863aff-c4b8-4ef2-93c9-3696cd9af787 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:14.586404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7863aff-c4b8-4ef2-93c9-3696cd9af787 service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:14.586688 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7863aff-c4b8-4ef2-93c9-3696cd9af787 service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:14.586964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7863aff-c4b8-4ef2-93c9-3696cd9af787 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:14.588653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f7863aff-c4b8-4ef2-93c9-3696cd9af787 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:14.685460 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7863aff-c4b8-4ef2-93c9-3696cd9af787 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:14.687125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7863aff-c4b8-4ef2-93c9-3696cd9af787 service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:14.706153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 710/1416] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:15.606979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb7d7b-64f1-4ed9-86d1-5eee74c899cf None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:15.607887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb7d7b-64f1-4ed9-86d1-5eee74c899cf None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:15.607887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb7d7b-64f1-4ed9-86d1-5eee74c899cf None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:15.609501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 05:37:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:15.616798 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec83db-391d-4d89-b1dd-0eeda575eb7a None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:15.617533 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec83db-391d-4d89-b1dd-0eeda575eb7a None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:15.617693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec83db-391d-4d89-b1dd-0eeda575eb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:15.707338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-6aec83db-391d-4d89-b1dd-0eeda575eb7a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:15.708572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-6aec83db-391d-4d89-b1dd-0eeda575eb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:15.736866 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:15.809141 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 05:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:15.968696 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb7d7b-64f1-4ed9-86d1-5eee74c899cf None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:15.970219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-fbdb7d7b-64f1-4ed9-86d1-5eee74c899cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:16.127524 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb7d7b-64f1-4ed9-86d1-5eee74c899cf None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:16.169531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:16.257219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a721b726-deca-4733-826e-cf13bdfe1b01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:16.257374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721b726-deca-4733-826e-cf13bdfe1b01 service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:16.257659 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721b726-deca-4733-826e-cf13bdfe1b01 service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:16.257954 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a721b726-deca-4733-826e-cf13bdfe1b01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:16.259595 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a721b726-deca-4733-826e-cf13bdfe1b01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:16.359744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721b726-deca-4733-826e-cf13bdfe1b01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:16.361725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a721b726-deca-4733-826e-cf13bdfe1b01 service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:16.381139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 708/1419] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:17.267005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af313452-41b6-4402-8720-3069d982a649 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:17.267876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af313452-41b6-4402-8720-3069d982a649 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:17.267973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af313452-41b6-4402-8720-3069d982a649 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:17.270146 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 05:37:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:17.278555 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426aff93-ef1c-45ec-9c6f-d8665a041e44 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:17.279000 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426aff93-ef1c-45ec-9c6f-d8665a041e44 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:17.279202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426aff93-ef1c-45ec-9c6f-d8665a041e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:17.340022 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:17.365054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-426aff93-ef1c-45ec-9c6f-d8665a041e44 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:17.366253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-426aff93-ef1c-45ec-9c6f-d8665a041e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:17.470949 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 05:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:17.624367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af313452-41b6-4402-8720-3069d982a649 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:17.625519 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-af313452-41b6-4402-8720-3069d982a649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:17.797123 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af313452-41b6-4402-8720-3069d982a649 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:17.809955 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:17.919532 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623f2b6-d34f-4adc-9b74-578c757710bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:17.920201 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623f2b6-d34f-4adc-9b74-578c757710bc service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:17.920480 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623f2b6-d34f-4adc-9b74-578c757710bc service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:17.920754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623f2b6-d34f-4adc-9b74-578c757710bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:17.922524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-1623f2b6-d34f-4adc-9b74-578c757710bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:18.021075 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623f2b6-d34f-4adc-9b74-578c757710bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:18.023056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623f2b6-d34f-4adc-9b74-578c757710bc service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:18.042118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 713/1422] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:18.896622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd14c53-57b6-456f-a1eb-cc9406bf2fa1 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:18.897541 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd14c53-57b6-456f-a1eb-cc9406bf2fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:18.897541 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd14c53-57b6-456f-a1eb-cc9406bf2fa1 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:18.899814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 05:37:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:18.907466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd4531-4e4a-498d-8fcb-6f0656bfd380 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:18.907891 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd4531-4e4a-498d-8fcb-6f0656bfd380 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:18.908132 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd4531-4e4a-498d-8fcb-6f0656bfd380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:18.927494 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:18.983720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-aafd4531-4e4a-498d-8fcb-6f0656bfd380 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:18.989588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-aafd4531-4e4a-498d-8fcb-6f0656bfd380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:19.098545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 05:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:19.258926 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd14c53-57b6-456f-a1eb-cc9406bf2fa1 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:19.261333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4cd14c53-57b6-456f-a1eb-cc9406bf2fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:19.380255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:19.417864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd14c53-57b6-456f-a1eb-cc9406bf2fa1 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:19.535363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9775e44-f0ea-4a93-b9e8-5e270a34b3f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:19.535936 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9775e44-f0ea-4a93-b9e8-5e270a34b3f3 service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:19.536218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9775e44-f0ea-4a93-b9e8-5e270a34b3f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:19.536493 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9775e44-f0ea-4a93-b9e8-5e270a34b3f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:19.538178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e9775e44-f0ea-4a93-b9e8-5e270a34b3f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:19.615961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:19.630749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9775e44-f0ea-4a93-b9e8-5e270a34b3f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:19.632397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9775e44-f0ea-4a93-b9e8-5e270a34b3f3 service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:19.650616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 711/1425] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:20.516131 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a28297-0532-43b1-b062-3dcf5945eba3 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:20.516970 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a28297-0532-43b1-b062-3dcf5945eba3 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:20.516970 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a28297-0532-43b1-b062-3dcf5945eba3 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:20.519085 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 05:37:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:20.525198 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bf31c-d13a-46a3-a15f-a4f1a5a293d0 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:20.525910 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bf31c-d13a-46a3-a15f-a4f1a5a293d0 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:20.526086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bf31c-d13a-46a3-a15f-a4f1a5a293d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:20.603592 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-344bf31c-d13a-46a3-a15f-a4f1a5a293d0 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:20.604782 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-344bf31c-d13a-46a3-a15f-a4f1a5a293d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:20.709566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 05:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:20.863374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a28297-0532-43b1-b062-3dcf5945eba3 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:20.865600 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-78a28297-0532-43b1-b062-3dcf5945eba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:20.960000 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:21.031360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a28297-0532-43b1-b062-3dcf5945eba3 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:21.173213 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4f403-82b3-4be1-a296-f1d0afa3966c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:21.174284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4f403-82b3-4be1-a296-f1d0afa3966c service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:21.174744 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4f403-82b3-4be1-a296-f1d0afa3966c service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:21.175286 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4f403-82b3-4be1-a296-f1d0afa3966c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:21.177973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-cda4f403-82b3-4be1-a296-f1d0afa3966c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:21.232290 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:21.282683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4f403-82b3-4be1-a296-f1d0afa3966c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:21.284335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4f403-82b3-4be1-a296-f1d0afa3966c service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:21.303160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 716/1428] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:22.100855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4de042-6d88-44e0-843a-325212bbfe2f None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:22.101837 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4de042-6d88-44e0-843a-325212bbfe2f None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:22.101837 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4de042-6d88-44e0-843a-325212bbfe2f None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:22.104004 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 05:37:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:22.109198 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96fedf-87e5-48e6-add8-3900608fe423 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:22.109758 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96fedf-87e5-48e6-add8-3900608fe423 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:22.109862 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96fedf-87e5-48e6-add8-3900608fe423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:22.187791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-ca96fedf-87e5-48e6-add8-3900608fe423 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:22.189001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ca96fedf-87e5-48e6-add8-3900608fe423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:22.287048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 05:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:22.437366 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4de042-6d88-44e0-843a-325212bbfe2f None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:22.438919 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-3b4de042-6d88-44e0-843a-325212bbfe2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:22.489266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:22.590577 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4de042-6d88-44e0-843a-325212bbfe2f None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:22.709889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9fdde9-f97b-40b3-99e6-05cf366f10bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:22.710473 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fdde9-f97b-40b3-99e6-05cf366f10bf service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:22.710758 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fdde9-f97b-40b3-99e6-05cf366f10bf service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:22.711056 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fdde9-f97b-40b3-99e6-05cf366f10bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:22.712719 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6b9fdde9-f97b-40b3-99e6-05cf366f10bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:22.720150 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:22.816202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9fdde9-f97b-40b3-99e6-05cf366f10bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:22.817831 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9fdde9-f97b-40b3-99e6-05cf366f10bf service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:22.837031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 714/1431] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:23.645311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b83d1-7322-42de-80a2-9937a0e7d2a7 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:23.645894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b83d1-7322-42de-80a2-9937a0e7d2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:23.645894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b83d1-7322-42de-80a2-9937a0e7d2a7 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:23.647266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 05:37:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:23.652529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668dc39-4c06-4412-8edc-2f2b45f5f879 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:23.652978 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668dc39-4c06-4412-8edc-2f2b45f5f879 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:23.653083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668dc39-4c06-4412-8edc-2f2b45f5f879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:23.739856 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-d668dc39-4c06-4412-8edc-2f2b45f5f879 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:23.741080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d668dc39-4c06-4412-8edc-2f2b45f5f879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:23.824685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:23.855591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 05:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:24.007797 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b83d1-7322-42de-80a2-9937a0e7d2a7 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:24.010031 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f27b83d1-7322-42de-80a2-9937a0e7d2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:24.033572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:24.177021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b83d1-7322-42de-80a2-9937a0e7d2a7 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:24.271660 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:24.297603 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1c62f-53a8-4c92-a800-a76e0b10115e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:24.298292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1c62f-53a8-4c92-a800-a76e0b10115e service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:24.298574 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1c62f-53a8-4c92-a800-a76e0b10115e service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:24.298852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1c62f-53a8-4c92-a800-a76e0b10115e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:24.300523 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-96c1c62f-53a8-4c92-a800-a76e0b10115e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:24.398687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c1c62f-53a8-4c92-a800-a76e0b10115e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:24.400342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c1c62f-53a8-4c92-a800-a76e0b10115e service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:24.420927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 719/1434] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:25.223667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6d777-bf55-4639-90d9-9f867e328c29 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:25.224165 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6d777-bf55-4639-90d9-9f867e328c29 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:25.224282 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6d777-bf55-4639-90d9-9f867e328c29 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:25.226311 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 05:37:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:25.231337 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a11b8-8216-4bbc-a505-514687e007ba None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:25.231750 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a11b8-8216-4bbc-a505-514687e007ba None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:25.231893 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a11b8-8216-4bbc-a505-514687e007ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:25.311393 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-b99a11b8-8216-4bbc-a505-514687e007ba None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:25.312593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b99a11b8-8216-4bbc-a505-514687e007ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:25.364971 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:25.431185 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 05:37:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:25.598713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6d777-bf55-4639-90d9-9f867e328c29 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:25.601114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-92c6d777-bf55-4639-90d9-9f867e328c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:25.773969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6d777-bf55-4639-90d9-9f867e328c29 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:25.837195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:25.904123 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c4bc3-be39-49ef-8f6b-d277f8e8b0da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:25.904780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c4bc3-be39-49ef-8f6b-d277f8e8b0da service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:25.905062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c4bc3-be39-49ef-8f6b-d277f8e8b0da service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:25.905332 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c4bc3-be39-49ef-8f6b-d277f8e8b0da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:25.906983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-117c4bc3-be39-49ef-8f6b-d277f8e8b0da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:26.007493 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c4bc3-be39-49ef-8f6b-d277f8e8b0da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:26.009141 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c4bc3-be39-49ef-8f6b-d277f8e8b0da service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:26.029242 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 717/1437] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:26.853164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a6b0c-fcc2-453a-a174-e30534b12972 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:26.853641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a6b0c-fcc2-453a-a174-e30534b12972 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:26.853739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a6b0c-fcc2-453a-a174-e30534b12972 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:26.856002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 05:37:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:26.863250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca029a-cfa5-4b6e-93ce-21a51154cc64 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:26.863639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca029a-cfa5-4b6e-93ce-21a51154cc64 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:26.863817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca029a-cfa5-4b6e-93ce-21a51154cc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:27.069324 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-5eca029a-cfa5-4b6e-93ce-21a51154cc64 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:27.070482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5eca029a-cfa5-4b6e-93ce-21a51154cc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:27.072965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:27.181575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 05:37:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:27.332017 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a6b0c-fcc2-453a-a174-e30534b12972 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:27.333445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-cb9a6b0c-fcc2-453a-a174-e30534b12972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:27.489294 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a6b0c-fcc2-453a-a174-e30534b12972 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:27.517183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:27.608332 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e2c80-f567-40fc-a921-b151ec99cc05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:27.608905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e2c80-f567-40fc-a921-b151ec99cc05 service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:27.609191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e2c80-f567-40fc-a921-b151ec99cc05 service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:27.609471 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e2c80-f567-40fc-a921-b151ec99cc05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:27.611254 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8d7e2c80-f567-40fc-a921-b151ec99cc05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:27.713012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e2c80-f567-40fc-a921-b151ec99cc05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:27.714643 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e2c80-f567-40fc-a921-b151ec99cc05 service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:27.732873 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 722/1440] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:29.660610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07458c9-d5fc-4af5-b392-4bbe54c20687 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:29.661193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07458c9-d5fc-4af5-b392-4bbe54c20687 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:29.661193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07458c9-d5fc-4af5-b392-4bbe54c20687 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:29.662901 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 05:37:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:29.670573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6f5f6-a637-4b56-8e82-149db10c9b76 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:29.671032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6f5f6-a637-4b56-8e82-149db10c9b76 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:29.671152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6f5f6-a637-4b56-8e82-149db10c9b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:29.710865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:29.738174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-7de6f5f6-a637-4b56-8e82-149db10c9b76 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:29.739423 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7de6f5f6-a637-4b56-8e82-149db10c9b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:29.841277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 05:37:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:37:29.887646 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07458c9-d5fc-4af5-b392-4bbe54c20687 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:29.889190 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a07458c9-d5fc-4af5-b392-4bbe54c20687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:30.004350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07458c9-d5fc-4af5-b392-4bbe54c20687 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:30.043273 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:30.087047 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139d780-7667-4c64-aa5a-30cd41316c34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:30.087727 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139d780-7667-4c64-aa5a-30cd41316c34 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:30.088012 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139d780-7667-4c64-aa5a-30cd41316c34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:30.088288 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139d780-7667-4c64-aa5a-30cd41316c34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:30.089923 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e139d780-7667-4c64-aa5a-30cd41316c34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:30.203572 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139d780-7667-4c64-aa5a-30cd41316c34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:30.205055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139d780-7667-4c64-aa5a-30cd41316c34 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:30.225587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 720/1443] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:32.402242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744653a-280f-4f7a-a97d-d0dba27bc604 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:32.403160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744653a-280f-4f7a-a97d-d0dba27bc604 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:32.403160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744653a-280f-4f7a-a97d-d0dba27bc604 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:32.405488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 05:37:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:32.411378 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bce42-fe60-4dfd-8b86-288e20e6fd9a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:32.411824 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bce42-fe60-4dfd-8b86-288e20e6fd9a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:32.411927 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bce42-fe60-4dfd-8b86-288e20e6fd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:32.478637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-836bce42-fe60-4dfd-8b86-288e20e6fd9a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:32.479892 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-836bce42-fe60-4dfd-8b86-288e20e6fd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:32.494614 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:32.588999 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 05:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:32.627578 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e744653a-280f-4f7a-a97d-d0dba27bc604 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:32.629125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-e744653a-280f-4f7a-a97d-d0dba27bc604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:32.737982 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e744653a-280f-4f7a-a97d-d0dba27bc604 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:32.750485 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:32.813897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0f0ab-bbae-4ebb-9853-fca80e4a7fc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:32.814553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0f0ab-bbae-4ebb-9853-fca80e4a7fc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:32.814839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0f0ab-bbae-4ebb-9853-fca80e4a7fc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:32.815159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0f0ab-bbae-4ebb-9853-fca80e4a7fc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:32.816804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-17c0f0ab-bbae-4ebb-9853-fca80e4a7fc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:32.926025 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c0f0ab-bbae-4ebb-9853-fca80e4a7fc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:32.927510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c0f0ab-bbae-4ebb-9853-fca80e4a7fc2 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:32.945877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 725/1446] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:35.408531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6cd3f7-8f03-403c-86a4-ea8a0fe53067 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:35.409106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6cd3f7-8f03-403c-86a4-ea8a0fe53067 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:35.409106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6cd3f7-8f03-403c-86a4-ea8a0fe53067 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:35.410745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 05:37:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:35.417415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d21288-eefa-4163-aef3-4c809adbcfd5 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:35.418106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d21288-eefa-4163-aef3-4c809adbcfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:35.418354 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d21288-eefa-4163-aef3-4c809adbcfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:35.474680 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:35.495416 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-71d21288-eefa-4163-aef3-4c809adbcfd5 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:35.496628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-71d21288-eefa-4163-aef3-4c809adbcfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:35.609581 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 05:37:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:35.652406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6cd3f7-8f03-403c-86a4-ea8a0fe53067 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:35.655044 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ba6cd3f7-8f03-403c-86a4-ea8a0fe53067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:35.766418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6cd3f7-8f03-403c-86a4-ea8a0fe53067 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:35.810942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:35.849372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2fc1d8-530a-4023-a22c-9247cb4b4647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:35.850023 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fc1d8-530a-4023-a22c-9247cb4b4647 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:35.850304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fc1d8-530a-4023-a22c-9247cb4b4647 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:35.850573 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2fc1d8-530a-4023-a22c-9247cb4b4647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:35.852248 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5a2fc1d8-530a-4023-a22c-9247cb4b4647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:35.969572 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fc1d8-530a-4023-a22c-9247cb4b4647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:35.971061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2fc1d8-530a-4023-a22c-9247cb4b4647 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:35.995214 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 723/1449] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:38.118508 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938a930-093e-4ce4-a09b-9e54525eea36 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:38.119107 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938a930-093e-4ce4-a09b-9e54525eea36 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:38.119107 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938a930-093e-4ce4-a09b-9e54525eea36 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:38.120404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 05:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:38.126888 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83da81-b291-4452-9a2c-88e6ffda64f7 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:38.127646 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83da81-b291-4452-9a2c-88e6ffda64f7 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:38.127870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83da81-b291-4452-9a2c-88e6ffda64f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:38.199639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-3a83da81-b291-4452-9a2c-88e6ffda64f7 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:38.200862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-3a83da81-b291-4452-9a2c-88e6ffda64f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:38.224826 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:38.316177 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 05:37:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:38.355538 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938a930-093e-4ce4-a09b-9e54525eea36 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:38.357906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f938a930-093e-4ce4-a09b-9e54525eea36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:38.479439 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938a930-093e-4ce4-a09b-9e54525eea36 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:38.500819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:38.555557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b780e39-e6f8-4614-81cb-619f8194a419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:38.556207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b780e39-e6f8-4614-81cb-619f8194a419 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:38.556484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b780e39-e6f8-4614-81cb-619f8194a419 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:38.556765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b780e39-e6f8-4614-81cb-619f8194a419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:38.558401 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9b780e39-e6f8-4614-81cb-619f8194a419 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:38.676589 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b780e39-e6f8-4614-81cb-619f8194a419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:38.678079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b780e39-e6f8-4614-81cb-619f8194a419 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:38.696160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 728/1452] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:40.777093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b4356-afb5-442e-9a07-4aa074ba1abf None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:40.777696 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b4356-afb5-442e-9a07-4aa074ba1abf None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:40.777696 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b4356-afb5-442e-9a07-4aa074ba1abf None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:40.779072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 05:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:40.785977 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f720e4-ae99-457e-bba3-45911a6f4a67 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:40.786729 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f720e4-ae99-457e-bba3-45911a6f4a67 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:40.786896 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f720e4-ae99-457e-bba3-45911a6f4a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:40.864266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:40.867588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-d1f720e4-ae99-457e-bba3-45911a6f4a67 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:40.869553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d1f720e4-ae99-457e-bba3-45911a6f4a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:40.983346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 05:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:41.021461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b4356-afb5-442e-9a07-4aa074ba1abf None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:41.022977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-da5b4356-afb5-442e-9a07-4aa074ba1abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:41.134424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b4356-afb5-442e-9a07-4aa074ba1abf None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:41.183725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:41.212383 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a9d819-e43d-42e8-8237-4b6c9f94b8a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:41.213043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9d819-e43d-42e8-8237-4b6c9f94b8a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:41.213322 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9d819-e43d-42e8-8237-4b6c9f94b8a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:41.213589 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a9d819-e43d-42e8-8237-4b6c9f94b8a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:41.215263 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-90a9d819-e43d-42e8-8237-4b6c9f94b8a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:41.332095 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9d819-e43d-42e8-8237-4b6c9f94b8a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:41.333465 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a9d819-e43d-42e8-8237-4b6c9f94b8a7 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:41.351756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 726/1455] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:37:43.424292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149ddc2-6616-4f7f-b740-c9621d72a319 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:43.424897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149ddc2-6616-4f7f-b740-c9621d72a319 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:43.424897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149ddc2-6616-4f7f-b740-c9621d72a319 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:43.426259 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 05:37:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:43.432700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b00d21-7b11-4d5e-92f4-0933834c9ac6 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:43.433205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b00d21-7b11-4d5e-92f4-0933834c9ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:43.433304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b00d21-7b11-4d5e-92f4-0933834c9ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:43.502664 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-c3b00d21-7b11-4d5e-92f4-0933834c9ac6 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:43.503909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c3b00d21-7b11-4d5e-92f4-0933834c9ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:43.532726 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:43.610727 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 05:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:43.650722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149ddc2-6616-4f7f-b740-c9621d72a319 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:43.652320 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-1149ddc2-6616-4f7f-b740-c9621d72a319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:43.772854 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149ddc2-6616-4f7f-b740-c9621d72a319 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:43.810581 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:43.858747 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e053d-15f7-4758-aed2-7d450ca7c784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:43.859474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e053d-15f7-4758-aed2-7d450ca7c784 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:43.859764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e053d-15f7-4758-aed2-7d450ca7c784 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:43.860046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e053d-15f7-4758-aed2-7d450ca7c784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:43.861782 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4d9e053d-15f7-4758-aed2-7d450ca7c784 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:43.974988 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e053d-15f7-4758-aed2-7d450ca7c784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:43.976470 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e053d-15f7-4758-aed2-7d450ca7c784 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:43.994759 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 731/1458] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:46.095199 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12defc04-f619-4e04-a4aa-33c49bcb0217 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:46.095759 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12defc04-f619-4e04-a4aa-33c49bcb0217 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:46.095759 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12defc04-f619-4e04-a4aa-33c49bcb0217 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:46.097060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 05:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:46.103626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fba8ef-ceef-4aba-8b3a-51014656d064 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:46.104422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fba8ef-ceef-4aba-8b3a-51014656d064 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:46.104584 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fba8ef-ceef-4aba-8b3a-51014656d064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:46.173224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-17fba8ef-ceef-4aba-8b3a-51014656d064 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:46.174421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-17fba8ef-ceef-4aba-8b3a-51014656d064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:46.183315 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:46.279574 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 05:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:46.318270 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12defc04-f619-4e04-a4aa-33c49bcb0217 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:46.320593 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-12defc04-f619-4e04-a4aa-33c49bcb0217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:46.436842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12defc04-f619-4e04-a4aa-33c49bcb0217 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:46.505852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:46.525003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ab23e-c0f6-4f91-b464-e88cef7ae848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:46.525659 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ab23e-c0f6-4f91-b464-e88cef7ae848 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:46.525972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ab23e-c0f6-4f91-b464-e88cef7ae848 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:46.526220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ab23e-c0f6-4f91-b464-e88cef7ae848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:46.527892 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ad4ab23e-c0f6-4f91-b464-e88cef7ae848 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:46.642528 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ab23e-c0f6-4f91-b464-e88cef7ae848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:46.644016 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ab23e-c0f6-4f91-b464-e88cef7ae848 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:46.663529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 729/1461] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:48.740127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54ed70-bed7-4c72-92a0-6877515062c6 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:48.740716 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54ed70-bed7-4c72-92a0-6877515062c6 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:48.740716 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54ed70-bed7-4c72-92a0-6877515062c6 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:48.742783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 05:37:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:48.751786 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ab15a-1371-430c-a9bf-203017642586 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:48.752187 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ab15a-1371-430c-a9bf-203017642586 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:48.752338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ab15a-1371-430c-a9bf-203017642586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:48.820363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-ec6ab15a-1371-430c-a9bf-203017642586 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:48.821588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ec6ab15a-1371-430c-a9bf-203017642586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:48.861004 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:48.925368 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 05:37:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:48.962487 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54ed70-bed7-4c72-92a0-6877515062c6 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:48.964075 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7f54ed70-bed7-4c72-92a0-6877515062c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:49.074182 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54ed70-bed7-4c72-92a0-6877515062c6 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:49.113367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:49.149190 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a06ad-7faa-452b-86ee-170e3f26923f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:49.149848 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a06ad-7faa-452b-86ee-170e3f26923f service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:49.150132 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a06ad-7faa-452b-86ee-170e3f26923f service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:49.150409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a06ad-7faa-452b-86ee-170e3f26923f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:49.152090 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-db6a06ad-7faa-452b-86ee-170e3f26923f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:49.267609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a06ad-7faa-452b-86ee-170e3f26923f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:49.269065 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a06ad-7faa-452b-86ee-170e3f26923f service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:49.287965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 734/1464] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:51.306043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fbbcf-5a0a-4ef6-8508-fcba08532a87 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:51.306650 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fbbcf-5a0a-4ef6-8508-fcba08532a87 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:51.306650 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fbbcf-5a0a-4ef6-8508-fcba08532a87 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:51.308016 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 05:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:51.313580 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bbdad-bc8d-422c-83d3-019a54f5b864 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:51.314026 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bbdad-bc8d-422c-83d3-019a54f5b864 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:51.314143 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bbdad-bc8d-422c-83d3-019a54f5b864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:51.382498 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-058bbdad-bc8d-422c-83d3-019a54f5b864 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:51.383720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-058bbdad-bc8d-422c-83d3-019a54f5b864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:51.401859 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:51.489242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 05:37:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:37:51.529677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047fbbcf-5a0a-4ef6-8508-fcba08532a87 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:51.532274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-047fbbcf-5a0a-4ef6-8508-fcba08532a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:51.650344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047fbbcf-5a0a-4ef6-8508-fcba08532a87 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:51.726167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:51.733846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eff1ed-fc7f-4414-b616-012cc186fecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:51.734506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eff1ed-fc7f-4414-b616-012cc186fecd service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:51.734790 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eff1ed-fc7f-4414-b616-012cc186fecd service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:51.735095 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eff1ed-fc7f-4414-b616-012cc186fecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:51.736738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-99eff1ed-fc7f-4414-b616-012cc186fecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:51.871520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eff1ed-fc7f-4414-b616-012cc186fecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:51.872979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eff1ed-fc7f-4414-b616-012cc186fecd service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:51.891430 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 732/1467] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:37:53.941968 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d1a43-8e0a-47fd-8610-a3dbe6f8471a None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:53.942881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d1a43-8e0a-47fd-8610-a3dbe6f8471a None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:53.942881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d1a43-8e0a-47fd-8610-a3dbe6f8471a None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:53.945037 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 05:37:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:37:53.953993 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d77b8d-ed4f-4612-bc7c-3db9c09a66ac None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:53.954685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d77b8d-ed4f-4612-bc7c-3db9c09a66ac None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:53.954863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d77b8d-ed4f-4612-bc7c-3db9c09a66ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:54.028952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-e1d77b8d-ed4f-4612-bc7c-3db9c09a66ac None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:37:54.030870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e1d77b8d-ed4f-4612-bc7c-3db9c09a66ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:54.082558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:54.135799 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 05:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:37:54.175602 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d1a43-8e0a-47fd-8610-a3dbe6f8471a None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:37:54.177188 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f93d1a43-8e0a-47fd-8610-a3dbe6f8471a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:54.287140 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d1a43-8e0a-47fd-8610-a3dbe6f8471a None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:37:54.336130 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:37:54.373028 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebbb3a-149b-4778-8209-2e8de16b26e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:37:54.373686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebbb3a-149b-4778-8209-2e8de16b26e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:37:54.374077 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebbb3a-149b-4778-8209-2e8de16b26e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:37:54.374356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebbb3a-149b-4778-8209-2e8de16b26e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:37:54.376045 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-22ebbb3a-149b-4778-8209-2e8de16b26e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:37:54.497353 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebbb3a-149b-4778-8209-2e8de16b26e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:37:54.498847 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebbb3a-149b-4778-8209-2e8de16b26e8 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:37:54.518140 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 737/1470] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:37:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:38:04.133718 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=22 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:38:04.515504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=66 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:38:14.086854 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e2193-273b-4dc0-a1cf-ea0edbb0a6b9 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:14.087781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e2193-273b-4dc0-a1cf-ea0edbb0a6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:14.087781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e2193-273b-4dc0-a1cf-ea0edbb0a6b9 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:14.090156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 734/1471] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:38:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:38:14.096488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d0d14-58e8-4072-a750-ac88140ebac0 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:14.096978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d0d14-58e8-4072-a750-ac88140ebac0 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:14.097094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d0d14-58e8-4072-a750-ac88140ebac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:14.174828 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-717d0d14-58e8-4072-a750-ac88140ebac0 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:38:14.176130 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-717d0d14-58e8-4072-a750-ac88140ebac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:14.277588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 738/1472] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:38:14.363087 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10892d-1010-4168-ae84-d0be05d846ba None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:14.363377 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10892d-1010-4168-ae84-d0be05d846ba None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:14.363663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10892d-1010-4168-ae84-d0be05d846ba None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:14.365059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 735/1473] 10.222.0.44 () {54 vars in 966 bytes} [Thu Jul 21 05:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:38:14.370260 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74315b00-ddfe-46a7-a234-87fa6d45685e None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:14.370554 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74315b00-ddfe-46a7-a234-87fa6d45685e None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:14.370831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74315b00-ddfe-46a7-a234-87fa6d45685e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:14.447695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-74315b00-ddfe-46a7-a234-87fa6d45685e None None] MFA Rules not processed for user `2707c6967a0a419a98ae142ff1d1a961`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:38:14.448892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-74315b00-ddfe-46a7-a234-87fa6d45685e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:14.474305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:38:14.544587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 739/1474] 10.222.0.44 () {58 vars in 1063 bytes} [Thu Jul 21 05:38:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:38:14.552351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10892d-1010-4168-ae84-d0be05d846ba None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:38:14.553873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ea10892d-1010-4168-ae84-d0be05d846ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:14.665741 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10892d-1010-4168-ae84-d0be05d846ba None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:38:14.741578 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7543db-8ff0-4c5e-a1fb-1864d8595d2c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2707c6967a0a419a98ae142ff1d1a961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:38:14.742235 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7543db-8ff0-4c5e-a1fb-1864d8595d2c service glance] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:14.742513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7543db-8ff0-4c5e-a1fb-1864d8595d2c service glance] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:14.742784 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7543db-8ff0-4c5e-a1fb-1864d8595d2c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:14.744454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ab7543db-8ff0-4c5e-a1fb-1864d8595d2c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:14.801458 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:38:14.859524 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7543db-8ff0-4c5e-a1fb-1864d8595d2c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:38:14.860978 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7543db-8ff0-4c5e-a1fb-1864d8595d2c service glance] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:38:14.880061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 736/1475] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:38:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:38:16.998602 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f816d-5992-40b3-bf4c-6846376722dd None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:16.999190 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f816d-5992-40b3-bf4c-6846376722dd None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:16.999190 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f816d-5992-40b3-bf4c-6846376722dd None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:17.000496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 740/1476] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:38:17.006033 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdb76a-a574-4478-b288-5559cf51767d None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:17.006489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdb76a-a574-4478-b288-5559cf51767d None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:17.006594 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdb76a-a574-4478-b288-5559cf51767d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:17.075234 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-abbdb76a-a574-4478-b288-5559cf51767d None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:38:17.076471 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-abbdb76a-a574-4478-b288-5559cf51767d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:17.187247 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 737/1477] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:38:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:38:17.212459 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f816d-5992-40b3-bf4c-6846376722dd None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:38:17.214801 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a00f816d-5992-40b3-bf4c-6846376722dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:17.353125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f816d-5992-40b3-bf4c-6846376722dd None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:38:17.403569 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:38:17.452197 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96928f39-4158-4691-b6a2-7469a6d760a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:38:17.452863 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96928f39-4158-4691-b6a2-7469a6d760a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:17.453146 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96928f39-4158-4691-b6a2-7469a6d760a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:17.453445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96928f39-4158-4691-b6a2-7469a6d760a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:17.454170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96928f39-4158-4691-b6a2-7469a6d760a4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:38:17.456023 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: WARNING py.warnings [None req-96928f39-4158-4691-b6a2-7469a6d760a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 05:38:17.456023 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: warnings.warn(msg) Jul 21 05:38:17.456023 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: Jul 21 05:38:17.457175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96928f39-4158-4691-b6a2-7469a6d760a4 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:38:17.477147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 741/1478] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jul 21 05:38:17 2022] GET /identity/v3/projects => generated 2553 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:38:19.526880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd8e2c-4d20-42b2-b4fb-41797154c98d None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:19.527865 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd8e2c-4d20-42b2-b4fb-41797154c98d None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:19.527865 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd8e2c-4d20-42b2-b4fb-41797154c98d None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:19.530041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 738/1479] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:38:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:38:19.537716 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2fb05-e3dd-4772-b8f6-e4230cc82787 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:19.538143 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2fb05-e3dd-4772-b8f6-e4230cc82787 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:19.538271 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2fb05-e3dd-4772-b8f6-e4230cc82787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:19.604228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-88e2fb05-e3dd-4772-b8f6-e4230cc82787 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:38:19.605436 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-88e2fb05-e3dd-4772-b8f6-e4230cc82787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:19.708140 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 742/1480] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:38:19.751836 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd8e2c-4d20-42b2-b4fb-41797154c98d None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:38:19.753788 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d9cd8e2c-4d20-42b2-b4fb-41797154c98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:19.766945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:38:19.913491 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd8e2c-4d20-42b2-b4fb-41797154c98d None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:38:19.996798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:38:20.030571 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea9577-8cbe-40da-99d7-0dd4db2282dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:38:20.031249 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea9577-8cbe-40da-99d7-0dd4db2282dd service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:20.031513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea9577-8cbe-40da-99d7-0dd4db2282dd service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:20.031807 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea9577-8cbe-40da-99d7-0dd4db2282dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:20.033447 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-98ea9577-8cbe-40da-99d7-0dd4db2282dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:20.126735 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea9577-8cbe-40da-99d7-0dd4db2282dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:38:20.128496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea9577-8cbe-40da-99d7-0dd4db2282dd service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:38:20.147313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 739/1481] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:38:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:38:22.361112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe38c2e-ac3a-4f0f-b55f-e3d8f713774d None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:22.361656 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe38c2e-ac3a-4f0f-b55f-e3d8f713774d None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:22.361656 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe38c2e-ac3a-4f0f-b55f-e3d8f713774d None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:22.363239 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 743/1482] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:38:22.368840 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643ccca-f9ab-448a-9d39-95b3459826ab None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:22.369368 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643ccca-f9ab-448a-9d39-95b3459826ab None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:22.369368 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643ccca-f9ab-448a-9d39-95b3459826ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:22.445440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-f643ccca-f9ab-448a-9d39-95b3459826ab None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:38:22.446672 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f643ccca-f9ab-448a-9d39-95b3459826ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:22.499003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:38:22.567362 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 740/1483] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:38:22.606935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe38c2e-ac3a-4f0f-b55f-e3d8f713774d None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:38:22.609332 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-afe38c2e-ac3a-4f0f-b55f-e3d8f713774d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:22.641091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:38:22.726018 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe38c2e-ac3a-4f0f-b55f-e3d8f713774d None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:38:22.813148 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c1970-7a4a-48c0-95a2-4706961cbb55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:38:22.813809 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c1970-7a4a-48c0-95a2-4706961cbb55 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:22.814085 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c1970-7a4a-48c0-95a2-4706961cbb55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:22.814361 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c1970-7a4a-48c0-95a2-4706961cbb55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:22.816052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9b3c1970-7a4a-48c0-95a2-4706961cbb55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:22.895302 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:38:22.941405 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c1970-7a4a-48c0-95a2-4706961cbb55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:38:22.942844 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c1970-7a4a-48c0-95a2-4706961cbb55 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:38:22.961029 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 744/1484] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:38:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:38:25.065354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c743d0-3f99-4aa9-9eb2-60c019310841 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:25.066233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c743d0-3f99-4aa9-9eb2-60c019310841 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:25.066233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c743d0-3f99-4aa9-9eb2-60c019310841 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:25.068342 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 741/1485] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:38:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:38:25.074726 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b2ef4-f6a7-4d5f-ac7d-4acb6ddfe8b1 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:25.075218 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b2ef4-f6a7-4d5f-ac7d-4acb6ddfe8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:25.075338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b2ef4-f6a7-4d5f-ac7d-4acb6ddfe8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:25.145511 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-ad7b2ef4-f6a7-4d5f-ac7d-4acb6ddfe8b1 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:38:25.146718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ad7b2ef4-f6a7-4d5f-ac7d-4acb6ddfe8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:25.256766 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 745/1486] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:38:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:38:25.295674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c743d0-3f99-4aa9-9eb2-60c019310841 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:38:25.297269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-59c743d0-3f99-4aa9-9eb2-60c019310841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:25.406123 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c743d0-3f99-4aa9-9eb2-60c019310841 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:38:25.491002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:38:25.493718 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944be661-f7be-4163-9aac-e44ac932f52f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:38:25.494634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944be661-f7be-4163-9aac-e44ac932f52f service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:25.494989 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944be661-f7be-4163-9aac-e44ac932f52f service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:25.495341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944be661-f7be-4163-9aac-e44ac932f52f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:25.498288 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-944be661-f7be-4163-9aac-e44ac932f52f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:25.612045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944be661-f7be-4163-9aac-e44ac932f52f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:38:25.613450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944be661-f7be-4163-9aac-e44ac932f52f service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:38:25.633367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 742/1487] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:38:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:38:27.866781 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0852af2-1f3e-4f35-bf71-17b064ccef3b None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:27.867376 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0852af2-1f3e-4f35-bf71-17b064ccef3b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:27.867702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0852af2-1f3e-4f35-bf71-17b064ccef3b None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:27.870177 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 746/1488] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:38:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:38:27.876084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8499a4b-cc90-4a26-9286-8d8febc6073a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:27.876570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8499a4b-cc90-4a26-9286-8d8febc6073a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:27.876674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8499a4b-cc90-4a26-9286-8d8febc6073a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:27.957471 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-a8499a4b-cc90-4a26-9286-8d8febc6073a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:38:27.958690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a8499a4b-cc90-4a26-9286-8d8febc6073a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:28.018546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:38:28.067404 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 743/1489] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:38:28.108742 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0852af2-1f3e-4f35-bf71-17b064ccef3b None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:38:28.110288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-c0852af2-1f3e-4f35-bf71-17b064ccef3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:28.150436 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:38:28.221223 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0852af2-1f3e-4f35-bf71-17b064ccef3b None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:38:28.296899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adefd17-7281-4a0a-a8da-dda5242024bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:38:28.297552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adefd17-7281-4a0a-a8da-dda5242024bb service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:28.297833 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adefd17-7281-4a0a-a8da-dda5242024bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:28.298112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adefd17-7281-4a0a-a8da-dda5242024bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:28.299784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5adefd17-7281-4a0a-a8da-dda5242024bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:28.383267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:38:28.421518 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adefd17-7281-4a0a-a8da-dda5242024bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:38:28.422974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adefd17-7281-4a0a-a8da-dda5242024bb service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:38:28.441978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 747/1490] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:38:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:38:30.697447 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da76d5-696b-4c9c-9a12-0c20aa4abe92 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:30.697971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da76d5-696b-4c9c-9a12-0c20aa4abe92 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:30.697971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da76d5-696b-4c9c-9a12-0c20aa4abe92 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:30.699429 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 744/1491] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jul 21 05:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:38:30.705078 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ca021-e61a-4a73-a4b8-07b4ef21ad8b None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:30.705500 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ca021-e61a-4a73-a4b8-07b4ef21ad8b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:30.705674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ca021-e61a-4a73-a4b8-07b4ef21ad8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:30.771178 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-170ca021-e61a-4a73-a4b8-07b4ef21ad8b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:38:30.772406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-170ca021-e61a-4a73-a4b8-07b4ef21ad8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:30.874190 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 748/1492] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jul 21 05:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:38:30.914644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da76d5-696b-4c9c-9a12-0c20aa4abe92 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:38:30.916234 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-56da76d5-696b-4c9c-9a12-0c20aa4abe92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:31.035132 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da76d5-696b-4c9c-9a12-0c20aa4abe92 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:38:31.114863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f23eb-f664-4b8a-947d-10364738379c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:38:31.115548 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f23eb-f664-4b8a-947d-10364738379c service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:38:31.115833 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f23eb-f664-4b8a-947d-10364738379c service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:38:31.116102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f23eb-f664-4b8a-947d-10364738379c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:38:31.117733 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-fc8f23eb-f664-4b8a-947d-10364738379c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:38:31.127222 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:38:31.240853 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f23eb-f664-4b8a-947d-10364738379c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:38:31.242325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f23eb-f664-4b8a-947d-10364738379c service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:38:31.260620 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 745/1493] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:38:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:38:40.872103 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=82 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:38:41.258229 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=50 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:39:57.008091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f644f1-dc33-4a29-8e03-abc621928ea4 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:39:57.008991 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f644f1-dc33-4a29-8e03-abc621928ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:39:57.008991 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f644f1-dc33-4a29-8e03-abc621928ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:39:57.102396 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-34f644f1-dc33-4a29-8e03-abc621928ea4 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:39:57.103671 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-34f644f1-dc33-4a29-8e03-abc621928ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:39:57.204734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 749/1494] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:39:57.212945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b09e5-a519-4ed9-a9a0-2e6abeb0215f None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:39:57.213491 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b09e5-a519-4ed9-a9a0-2e6abeb0215f None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:39:57.213491 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b09e5-a519-4ed9-a9a0-2e6abeb0215f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:39:57.290320 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-1f7b09e5-a519-4ed9-a9a0-2e6abeb0215f None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:39:57.291575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1f7b09e5-a519-4ed9-a9a0-2e6abeb0215f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:39:57.401007 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 746/1495] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:39:57.406039 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f644f1-dc33-4a29-8e03-abc621928ea4 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:39:57.407295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-34f644f1-dc33-4a29-8e03-abc621928ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:39:57.419438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f644f1-dc33-4a29-8e03-abc621928ea4 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:39:57.491434 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:39:57.527301 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7f942-5f7b-4c35-bee1-5b5c2e81775a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:39:57.528020 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7f942-5f7b-4c35-bee1-5b5c2e81775a admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:39:57.528303 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7f942-5f7b-4c35-bee1-5b5c2e81775a admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:39:57.528612 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7f942-5f7b-4c35-bee1-5b5c2e81775a admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:39:57.529342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7f942-5f7b-4c35-bee1-5b5c2e81775a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:39:57.529654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7f942-5f7b-4c35-bee1-5b5c2e81775a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:39:57.530694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: WARNING py.warnings [None req-5fa7f942-5f7b-4c35-bee1-5b5c2e81775a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 05:39:57.530694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: warnings.warn(msg) Jul 21 05:39:57.530694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: Jul 21 05:39:57.531761 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7f942-5f7b-4c35-bee1-5b5c2e81775a admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:39:57.543985 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 750/1496] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:39:57.552675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b09e5-a519-4ed9-a9a0-2e6abeb0215f None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:39:57.562081 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b09e5-a519-4ed9-a9a0-2e6abeb0215f None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:39:57.624930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:39:57.661866 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624f694-71c0-4281-8529-625d57c83314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:39:57.663204 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624f694-71c0-4281-8529-625d57c83314 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:39:57.663204 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624f694-71c0-4281-8529-625d57c83314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:39:57.663204 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624f694-71c0-4281-8529-625d57c83314 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:39:57.663842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624f694-71c0-4281-8529-625d57c83314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:39:57.665600 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: WARNING py.warnings [None req-f624f694-71c0-4281-8529-625d57c83314 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 05:39:57.665600 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: warnings.warn(msg) Jul 21 05:39:57.665600 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: Jul 21 05:39:57.666808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624f694-71c0-4281-8529-625d57c83314 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:39:57.723147 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 747/1497] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:39:57 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:39:57.728246 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7f942-5f7b-4c35-bee1-5b5c2e81775a admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:39:57.729872 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5fa7f942-5f7b-4c35-bee1-5b5c2e81775a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:39:57.772564 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7f942-5f7b-4c35-bee1-5b5c2e81775a admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:39:57.881652 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccca99-d1da-4041-ab68-65a7d4a34682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:39:57.882211 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccca99-d1da-4041-ab68-65a7d4a34682 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:39:57.882489 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccca99-d1da-4041-ab68-65a7d4a34682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:39:57.882760 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccca99-d1da-4041-ab68-65a7d4a34682 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:39:57.883479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccca99-d1da-4041-ab68-65a7d4a34682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:39:57.884507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: WARNING py.warnings [None req-0fccca99-d1da-4041-ab68-65a7d4a34682 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 05:39:57.884507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: warnings.warn(msg) Jul 21 05:39:57.884507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: Jul 21 05:39:57.885735 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccca99-d1da-4041-ab68-65a7d4a34682 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:39:57.928628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 751/1498] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:39:57 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:39:57.933789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624f694-71c0-4281-8529-625d57c83314 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:39:57.936277 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f624f694-71c0-4281-8529-625d57c83314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:39:58.000412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624f694-71c0-4281-8529-625d57c83314 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:39:58.095270 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:39:58.108216 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71d276-c542-41ea-b157-9828e864a2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:39:58.108898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71d276-c542-41ea-b157-9828e864a2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:39:58.109178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71d276-c542-41ea-b157-9828e864a2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:39:58.109451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa71d276-c542-41ea-b157-9828e864a2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:39:58.110177 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71d276-c542-41ea-b157-9828e864a2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:39:58.111328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: WARNING py.warnings [None req-aa71d276-c542-41ea-b157-9828e864a2e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 05:39:58.111328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: warnings.warn(msg) Jul 21 05:39:58.111328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: Jul 21 05:39:58.112218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa71d276-c542-41ea-b157-9828e864a2e0 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:39:58.124967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 748/1499] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:39:58.129324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccca99-d1da-4041-ab68-65a7d4a34682 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:39:58.130851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0fccca99-d1da-4041-ab68-65a7d4a34682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:39:58.142729 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccca99-d1da-4041-ab68-65a7d4a34682 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:39:58.169361 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:39:58.239862 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ab856-3517-4a59-b645-a3b4e8880add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:39:58.240542 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ab856-3517-4a59-b645-a3b4e8880add admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:39:58.240824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ab856-3517-4a59-b645-a3b4e8880add admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:39:58.241102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ab856-3517-4a59-b645-a3b4e8880add admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:39:58.241834 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ab856-3517-4a59-b645-a3b4e8880add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:39:58.242890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: WARNING py.warnings [None req-5d5ab856-3517-4a59-b645-a3b4e8880add admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 05:39:58.242890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: warnings.warn(msg) Jul 21 05:39:58.242890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: Jul 21 05:39:58.243820 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ab856-3517-4a59-b645-a3b4e8880add admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:39:58.257098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 752/1500] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:39:58.261899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71d276-c542-41ea-b157-9828e864a2e0 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:39:58.263571 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-aa71d276-c542-41ea-b157-9828e864a2e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:39:58.298541 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa71d276-c542-41ea-b157-9828e864a2e0 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:39:58.396806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81167bb7-721f-4df7-a94a-4911f6dd3d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:39:58.397635 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81167bb7-721f-4df7-a94a-4911f6dd3d76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:39:58.397915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81167bb7-721f-4df7-a94a-4911f6dd3d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:39:58.398187 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81167bb7-721f-4df7-a94a-4911f6dd3d76 admin admin] PATH_INFO: `/v3/projects/be1efe4672ad441281dc2ad891cbb25a/users/3750745ad7464c7a9ef854ca0d468c3f/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:39:58.420754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81167bb7-721f-4df7-a94a-4911f6dd3d76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be1efe4672ad441281dc2ad891cbb25a, user_id=3750745ad7464c7a9ef854ca0d468c3f, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:39:58.421786 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: WARNING py.warnings [None req-81167bb7-721f-4df7-a94a-4911f6dd3d76 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 05:39:58.421786 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: warnings.warn(msg) Jul 21 05:39:58.421786 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: Jul 21 05:39:58.423823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81167bb7-721f-4df7-a94a-4911f6dd3d76 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:39:58.450108 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 749/1501] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:39:58 2022] PUT /identity/v3/projects/be1efe4672ad441281dc2ad891cbb25a/users/3750745ad7464c7a9ef854ca0d468c3f/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:39:58.462527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da006b1-a8b3-4cec-a81a-eb67cd178c49 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:39:58.462842 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da006b1-a8b3-4cec-a81a-eb67cd178c49 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:39:58.463156 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da006b1-a8b3-4cec-a81a-eb67cd178c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:39:58.521478 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-4da006b1-a8b3-4cec-a81a-eb67cd178c49 None None] MFA Rules not processed for user `3750745ad7464c7a9ef854ca0d468c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:39:58.522677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4da006b1-a8b3-4cec-a81a-eb67cd178c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:39:58.605941 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 753/1502] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:39:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:39:58.621474 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73112e-74aa-4544-be76-83f86dea3478 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:39:58.621797 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73112e-74aa-4544-be76-83f86dea3478 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:39:58.622156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73112e-74aa-4544-be76-83f86dea3478 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:39:58.624443 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 750/1503] 10.222.0.44 () {54 vars in 964 bytes} [Thu Jul 21 05:39:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:39:58.629510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ae9f3-38c4-4b50-8ad2-d08c2ad0deaa None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:39:58.629796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ae9f3-38c4-4b50-8ad2-d08c2ad0deaa None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:39:58.630069 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ae9f3-38c4-4b50-8ad2-d08c2ad0deaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:39:58.660415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:39:58.709270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-988ae9f3-38c4-4b50-8ad2-d08c2ad0deaa None None] MFA Rules not processed for user `f3f76371c6a04fc5a477d6ef1908a08e`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:39:58.710498 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-988ae9f3-38c4-4b50-8ad2-d08c2ad0deaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:39:58.790399 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 754/1504] 10.222.0.44 () {58 vars in 1061 bytes} [Thu Jul 21 05:39:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:39:58.798613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73112e-74aa-4544-be76-83f86dea3478 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:39:58.800247 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8c73112e-74aa-4544-be76-83f86dea3478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:39:58.852405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:39:58.906063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73112e-74aa-4544-be76-83f86dea3478 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:39:58.983264 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebcfccb-9bca-4874-bd07-9f91673a48eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:39:58.983847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcfccb-9bca-4874-bd07-9f91673a48eb service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:39:58.984133 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcfccb-9bca-4874-bd07-9f91673a48eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:39:58.984411 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcfccb-9bca-4874-bd07-9f91673a48eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:39:58.986129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4ebcfccb-9bca-4874-bd07-9f91673a48eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:39:59.088467 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebcfccb-9bca-4874-bd07-9f91673a48eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:39:59.089916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebcfccb-9bca-4874-bd07-9f91673a48eb service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:39:59.093452 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:39:59.111930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 751/1505] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:39:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:39:59.154203 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d8097-a7b0-4e35-ba3d-b6d77f087f55 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:39:59.154654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d8097-a7b0-4e35-ba3d-b6d77f087f55 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:39:59.154769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d8097-a7b0-4e35-ba3d-b6d77f087f55 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:39:59.156120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 755/1506] 10.222.0.44 () {52 vars in 829 bytes} [Thu Jul 21 05:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:39:59.160144 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebcfccb-9bca-4874-bd07-9f91673a48eb service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:39:59.161775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4ebcfccb-9bca-4874-bd07-9f91673a48eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:39:59.271639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebcfccb-9bca-4874-bd07-9f91673a48eb service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:39:59.377699 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cff03-80db-4dd4-9378-e3e3b1a5dc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:39:59.378635 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cff03-80db-4dd4-9378-e3e3b1a5dc19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:39:59.378916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cff03-80db-4dd4-9378-e3e3b1a5dc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:39:59.379221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6cff03-80db-4dd4-9378-e3e3b1a5dc19 admin admin] PATH_INFO: `/v3/users/3750745ad7464c7a9ef854ca0d468c3f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:39:59.392043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cff03-80db-4dd4-9378-e3e3b1a5dc19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3750745ad7464c7a9ef854ca0d468c3f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:39:59.393088 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: WARNING py.warnings [None req-5a6cff03-80db-4dd4-9378-e3e3b1a5dc19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 05:39:59.393088 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: warnings.warn(msg) Jul 21 05:39:59.393088 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: Jul 21 05:39:59.394313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6cff03-80db-4dd4-9378-e3e3b1a5dc19 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:39:59.404242 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:39:59.462512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5a6cff03-80db-4dd4-9378-e3e3b1a5dc19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3750745ad7464c7a9ef854ca0d468c3f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:39:59.481490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5a6cff03-80db-4dd4-9378-e3e3b1a5dc19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3750745ad7464c7a9ef854ca0d468c3f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:39:59.497729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5a6cff03-80db-4dd4-9378-e3e3b1a5dc19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3750745ad7464c7a9ef854ca0d468c3f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:39:59.498288 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5a6cff03-80db-4dd4-9378-e3e3b1a5dc19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3750745ad7464c7a9ef854ca0d468c3f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:39:59.498778 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5a6cff03-80db-4dd4-9378-e3e3b1a5dc19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3750745ad7464c7a9ef854ca0d468c3f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:39:59.522174 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 752/1507] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:39:59 2022] DELETE /identity/v3/users/3750745ad7464c7a9ef854ca0d468c3f => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:39:59.526829 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2b90f-5873-4d95-abd8-e7b53ee6345e None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:39:59.527145 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2b90f-5873-4d95-abd8-e7b53ee6345e None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:39:59.527446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2b90f-5873-4d95-abd8-e7b53ee6345e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:39:59.583782 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-a3c2b90f-5873-4d95-abd8-e7b53ee6345e None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:39:59.584994 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a3c2b90f-5873-4d95-abd8-e7b53ee6345e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:39:59.663360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:39:59.677263 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 756/1508] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:39:59.687849 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cff03-80db-4dd4-9378-e3e3b1a5dc19 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:39:59.689099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5a6cff03-80db-4dd4-9378-e3e3b1a5dc19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:39:59.793670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6cff03-80db-4dd4-9378-e3e3b1a5dc19 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:39:59.863979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:39:59.886465 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebc445-035a-41e4-8bc3-26849f91b8d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:39:59.887162 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebc445-035a-41e4-8bc3-26849f91b8d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:39:59.887448 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebc445-035a-41e4-8bc3-26849f91b8d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:39:59.887718 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebc445-035a-41e4-8bc3-26849f91b8d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:39:59.889361 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a1ebc445-035a-41e4-8bc3-26849f91b8d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:40:00.008792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ebc445-035a-41e4-8bc3-26849f91b8d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:40:00.010259 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ebc445-035a-41e4-8bc3-26849f91b8d5 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:40:00.030952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 753/1509] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:39:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:40:00.404848 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2b90f-5873-4d95-abd8-e7b53ee6345e None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:40:00.406113 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a3c2b90f-5873-4d95-abd8-e7b53ee6345e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:40:00.418004 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2b90f-5873-4d95-abd8-e7b53ee6345e None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:40:00.520491 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e186b1-fbb8-48fc-adf0-7a6408dc451b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:40:00.521413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e186b1-fbb8-48fc-adf0-7a6408dc451b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:40:00.521694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e186b1-fbb8-48fc-adf0-7a6408dc451b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:40:00.521973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e186b1-fbb8-48fc-adf0-7a6408dc451b admin admin] PATH_INFO: `/v3/projects/be1efe4672ad441281dc2ad891cbb25a` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:40:00.532785 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e186b1-fbb8-48fc-adf0-7a6408dc451b admin admin] RBAC: Authorizing `identity:delete_project(project_id=be1efe4672ad441281dc2ad891cbb25a)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:40:00.533830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: WARNING py.warnings [None req-28e186b1-fbb8-48fc-adf0-7a6408dc451b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 05:40:00.533830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: warnings.warn(msg) Jul 21 05:40:00.533830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: Jul 21 05:40:00.535076 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e186b1-fbb8-48fc-adf0-7a6408dc451b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:40:00.594817 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-28e186b1-fbb8-48fc-adf0-7a6408dc451b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be1efe4672ad441281dc2ad891cbb25a', 'request_id': 'req-28e186b1-fbb8-48fc-adf0-7a6408dc451b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:40:00.608273 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-28e186b1-fbb8-48fc-adf0-7a6408dc451b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be1efe4672ad441281dc2ad891cbb25a', 'request_id': 'req-28e186b1-fbb8-48fc-adf0-7a6408dc451b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:40:00.616166 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-28e186b1-fbb8-48fc-adf0-7a6408dc451b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be1efe4672ad441281dc2ad891cbb25a', 'request_id': 'req-28e186b1-fbb8-48fc-adf0-7a6408dc451b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:40:00.649375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-28e186b1-fbb8-48fc-adf0-7a6408dc451b admin admin] The token cache is being invalidate because project be1efe4672ad441281dc2ad891cbb25a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:40:00.649672 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-28e186b1-fbb8-48fc-adf0-7a6408dc451b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:40:00.650240 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-28e186b1-fbb8-48fc-adf0-7a6408dc451b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:40:00.651669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 757/1510] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:40:00 2022] DELETE /identity/v3/projects/be1efe4672ad441281dc2ad891cbb25a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:40:10.028583 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=62 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:40:10.646997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:41:38.942993 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfae0f-ad66-47e1-9ab2-eac5bf7825bb None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:38.943657 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfae0f-ad66-47e1-9ab2-eac5bf7825bb None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:38.943657 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becfae0f-ad66-47e1-9ab2-eac5bf7825bb None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:38.944997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 05:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:41:38.951232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bf2f7-e35c-406f-a2e6-7ca869d07929 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:38.951992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bf2f7-e35c-406f-a2e6-7ca869d07929 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:38.952155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589bf2f7-e35c-406f-a2e6-7ca869d07929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:39.166773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-589bf2f7-e35c-406f-a2e6-7ca869d07929 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:41:39.168090 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-589bf2f7-e35c-406f-a2e6-7ca869d07929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:41:39.275373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 05:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:41:39.435322 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfae0f-ad66-47e1-9ab2-eac5bf7825bb None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:41:39.439294 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-becfae0f-ad66-47e1-9ab2-eac5bf7825bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:41:39.570978 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becfae0f-ad66-47e1-9ab2-eac5bf7825bb None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:41:39.576840 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff97aa-4c48-4298-87f5-741b14a7dfb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:41:39.577500 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff97aa-4c48-4298-87f5-741b14a7dfb5 service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:39.577765 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff97aa-4c48-4298-87f5-741b14a7dfb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:39.578040 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff97aa-4c48-4298-87f5-741b14a7dfb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:39.586249 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff97aa-4c48-4298-87f5-741b14a7dfb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:41:39.587892 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff97aa-4c48-4298-87f5-741b14a7dfb5 service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:41:39.597335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 755/1513] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:41:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:41:40.422851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14bdbe-e5f6-41ca-8924-143eb315183a None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:40.423507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14bdbe-e5f6-41ca-8924-143eb315183a None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:40.423507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14bdbe-e5f6-41ca-8924-143eb315183a None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:40.424852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 05:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:41:40.430119 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14683e06-a7d7-40bc-bd51-7ea4d9c4731b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:40.430573 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14683e06-a7d7-40bc-bd51-7ea4d9c4731b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:40.430666 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14683e06-a7d7-40bc-bd51-7ea4d9c4731b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:40.455327 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-14683e06-a7d7-40bc-bd51-7ea4d9c4731b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:41:40.456524 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-14683e06-a7d7-40bc-bd51-7ea4d9c4731b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:41:40.465480 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 05:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:41:40.617408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14bdbe-e5f6-41ca-8924-143eb315183a None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:41:40.633726 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14bdbe-e5f6-41ca-8924-143eb315183a None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:41:40.640139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd10e5d-6db7-4db1-aead-68cc04a54f36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:41:40.640820 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd10e5d-6db7-4db1-aead-68cc04a54f36 service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:40.641108 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd10e5d-6db7-4db1-aead-68cc04a54f36 service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:40.641371 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd10e5d-6db7-4db1-aead-68cc04a54f36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:40.650157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd10e5d-6db7-4db1-aead-68cc04a54f36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:41:40.651838 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd10e5d-6db7-4db1-aead-68cc04a54f36 service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:41:40.662803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 760/1516] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:41:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:41:40.961350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3ac88-f229-41b9-8776-a8d526bc0f54 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:40.961582 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3ac88-f229-41b9-8776-a8d526bc0f54 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:40.961805 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3ac88-f229-41b9-8776-a8d526bc0f54 None None] PATH_INFO: `/v3` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:40.963221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 757/1517] 10.222.0.47 () {54 vars in 925 bytes} [Thu Jul 21 05:41:40 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:41:40.971250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a707e3-97bb-40b7-886d-0b19d7be5e2c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:40.971543 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a707e3-97bb-40b7-886d-0b19d7be5e2c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:40.971820 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a707e3-97bb-40b7-886d-0b19d7be5e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:41.032752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-48a707e3-97bb-40b7-886d-0b19d7be5e2c None None] MFA Rules not processed for user `03d868598e39448881b43d299253d821`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:41:41.033964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-48a707e3-97bb-40b7-886d-0b19d7be5e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:41:41.052789 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:41:41.109623 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 761/1518] 10.222.0.47 () {58 vars in 1015 bytes} [Thu Jul 21 05:41:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:41:50.101769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb18efb-d867-4eec-a48c-313f941ba7f4 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:50.102325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb18efb-d867-4eec-a48c-313f941ba7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:50.102325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb18efb-d867-4eec-a48c-313f941ba7f4 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:50.103727 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 758/1519] 10.222.0.44 () {54 vars in 961 bytes} [Thu Jul 21 05:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:41:50.109769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271cd07-89a4-4583-93b5-bb2b1f46f0cd None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:50.110200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271cd07-89a4-4583-93b5-bb2b1f46f0cd None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:50.110310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271cd07-89a4-4583-93b5-bb2b1f46f0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:50.131576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-7271cd07-89a4-4583-93b5-bb2b1f46f0cd None None] MFA Rules not processed for user `03d868598e39448881b43d299253d821`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:41:50.132779 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7271cd07-89a4-4583-93b5-bb2b1f46f0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:41:50.142079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 762/1520] 10.222.0.44 () {58 vars in 1058 bytes} [Thu Jul 21 05:41:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:41:50.149387 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb18efb-d867-4eec-a48c-313f941ba7f4 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:41:50.164646 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb18efb-d867-4eec-a48c-313f941ba7f4 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:41:50.172800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e472a-c6af-422e-a431-cb0d251888a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:41:50.173880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e472a-c6af-422e-a431-cb0d251888a9 service placement] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:50.174373 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e472a-c6af-422e-a431-cb0d251888a9 service placement] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:50.174818 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e472a-c6af-422e-a431-cb0d251888a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:50.189842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e472a-c6af-422e-a431-cb0d251888a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:41:50.192555 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e472a-c6af-422e-a431-cb0d251888a9 service placement] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:41:50.204372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 759/1521] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:41:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:41:51.903728 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695d9d2-dfda-4e94-86ae-1545ca86081d None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:51.904296 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695d9d2-dfda-4e94-86ae-1545ca86081d None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:51.904296 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695d9d2-dfda-4e94-86ae-1545ca86081d None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:51.905682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 05:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:41:51.911810 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92318d4-7f46-42be-91a5-53dc5d2c991b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:51.912277 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92318d4-7f46-42be-91a5-53dc5d2c991b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:51.912398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92318d4-7f46-42be-91a5-53dc5d2c991b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:51.939543 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-a92318d4-7f46-42be-91a5-53dc5d2c991b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:41:51.940664 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a92318d4-7f46-42be-91a5-53dc5d2c991b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:41:51.954314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 05:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:41:52.108181 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695d9d2-dfda-4e94-86ae-1545ca86081d None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:41:52.131172 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695d9d2-dfda-4e94-86ae-1545ca86081d None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:41:52.138315 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3906c-b143-453c-9c19-9a97cf84aecd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:41:52.138956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3906c-b143-453c-9c19-9a97cf84aecd service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:52.139318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3906c-b143-453c-9c19-9a97cf84aecd service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:52.139594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3906c-b143-453c-9c19-9a97cf84aecd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:52.147963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3906c-b143-453c-9c19-9a97cf84aecd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:41:52.149556 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3906c-b143-453c-9c19-9a97cf84aecd service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:41:52.159038 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 764/1524] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:41:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:41:52.987951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70ddbc-8225-4579-97c5-30a2105766d0 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:52.988526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70ddbc-8225-4579-97c5-30a2105766d0 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:52.988526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70ddbc-8225-4579-97c5-30a2105766d0 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:52.989801 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 21 05:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:41:52.995966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ee4d3-0cdd-486a-bb80-47d00197ac29 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:52.996754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ee4d3-0cdd-486a-bb80-47d00197ac29 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:52.996924 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ee4d3-0cdd-486a-bb80-47d00197ac29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:53.021183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-429ee4d3-0cdd-486a-bb80-47d00197ac29 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:41:53.022376 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-429ee4d3-0cdd-486a-bb80-47d00197ac29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:41:53.031257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 21 05:41:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:41:53.183842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70ddbc-8225-4579-97c5-30a2105766d0 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:41:53.199790 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70ddbc-8225-4579-97c5-30a2105766d0 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:41:53.205700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a177843-41d3-4eb9-b6e4-928c1c0d4fa9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:41:53.206364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a177843-41d3-4eb9-b6e4-928c1c0d4fa9 service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:53.206644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a177843-41d3-4eb9-b6e4-928c1c0d4fa9 service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:53.206912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a177843-41d3-4eb9-b6e4-928c1c0d4fa9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:53.215343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a177843-41d3-4eb9-b6e4-928c1c0d4fa9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:41:53.216919 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a177843-41d3-4eb9-b6e4-928c1c0d4fa9 service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:41:53.226373 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 762/1527] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:41:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:41:57.036237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3b503-f306-4bc8-b440-2990155733ad None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:57.036854 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3b503-f306-4bc8-b440-2990155733ad None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:57.036854 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3b503-f306-4bc8-b440-2990155733ad None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:57.038198 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 05:41:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:41:57.043951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f3fbf-386b-45f8-83d6-a5a38ac26f7a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:57.044427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f3fbf-386b-45f8-83d6-a5a38ac26f7a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:57.044530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f3fbf-386b-45f8-83d6-a5a38ac26f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:57.063296 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-151f3fbf-386b-45f8-83d6-a5a38ac26f7a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:41:57.064494 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-151f3fbf-386b-45f8-83d6-a5a38ac26f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:41:57.072354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 05:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:41:57.150225 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3b503-f306-4bc8-b440-2990155733ad None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:41:57.153366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0fb3b503-f306-4bc8-b440-2990155733ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:41:57.247413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3b503-f306-4bc8-b440-2990155733ad None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:41:57.252364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b81efc-0caf-43a2-824f-fb795cc0c861 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:41:57.253016 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b81efc-0caf-43a2-824f-fb795cc0c861 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:57.253294 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b81efc-0caf-43a2-824f-fb795cc0c861 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:57.253617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b81efc-0caf-43a2-824f-fb795cc0c861 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:57.263308 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b81efc-0caf-43a2-824f-fb795cc0c861 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:41:57.264740 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b81efc-0caf-43a2-824f-fb795cc0c861 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:41:57.274251 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 767/1530] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:41:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:41:59.278039 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eade16-eefc-4e11-8154-c283aab2d007 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:59.278705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eade16-eefc-4e11-8154-c283aab2d007 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:59.278705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eade16-eefc-4e11-8154-c283aab2d007 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:59.280149 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 05:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:41:59.286043 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7786c6-1628-479c-b1ac-b18a43e6d2a7 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:59.286452 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7786c6-1628-479c-b1ac-b18a43e6d2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:59.286591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7786c6-1628-479c-b1ac-b18a43e6d2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:59.306139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-3f7786c6-1628-479c-b1ac-b18a43e6d2a7 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:41:59.307352 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3f7786c6-1628-479c-b1ac-b18a43e6d2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:41:59.316025 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 05:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:41:59.352090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eade16-eefc-4e11-8154-c283aab2d007 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:41:59.365609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eade16-eefc-4e11-8154-c283aab2d007 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:41:59.369972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfc578-f4e2-4a78-b909-7c1bad5ce0f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:41:59.370632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfc578-f4e2-4a78-b909-7c1bad5ce0f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:41:59.370910 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfc578-f4e2-4a78-b909-7c1bad5ce0f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:41:59.371238 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfc578-f4e2-4a78-b909-7c1bad5ce0f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:41:59.381848 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cfc578-f4e2-4a78-b909-7c1bad5ce0f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:41:59.384042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cfc578-f4e2-4a78-b909-7c1bad5ce0f0 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:41:59.399621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 765/1533] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:41:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:42:02.893502 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081f2b8-438e-4cb4-8533-e233677fe398 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:42:02.894044 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081f2b8-438e-4cb4-8533-e233677fe398 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:42:02.894044 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081f2b8-438e-4cb4-8533-e233677fe398 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:42:02.895385 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 05:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:42:02.900839 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f09587-fba6-47d1-98ea-ef25bc4d6719 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:42:02.901289 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f09587-fba6-47d1-98ea-ef25bc4d6719 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:42:02.901394 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f09587-fba6-47d1-98ea-ef25bc4d6719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:42:02.908227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=81 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:42:02.921703 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-b9f09587-fba6-47d1-98ea-ef25bc4d6719 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:42:02.922884 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b9f09587-fba6-47d1-98ea-ef25bc4d6719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:42:02.931231 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 05:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:42:03.239494 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081f2b8-438e-4cb4-8533-e233677fe398 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:42:03.242571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0081f2b8-438e-4cb4-8533-e233677fe398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:42:03.317212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:42:03.332030 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081f2b8-438e-4cb4-8533-e233677fe398 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:42:03.335721 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38196846-8a6a-4fea-8c54-46189509015d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:42:03.336400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38196846-8a6a-4fea-8c54-46189509015d service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:42:03.336676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38196846-8a6a-4fea-8c54-46189509015d service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:42:03.336956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38196846-8a6a-4fea-8c54-46189509015d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:42:03.346253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38196846-8a6a-4fea-8c54-46189509015d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:42:03.347724 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38196846-8a6a-4fea-8c54-46189509015d service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:42:03.357312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 770/1536] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:42:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:42:05.424635 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841e157-23e0-49ed-ba84-1b48925bae9c None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:42:05.425269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841e157-23e0-49ed-ba84-1b48925bae9c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:42:05.425269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841e157-23e0-49ed-ba84-1b48925bae9c None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:42:05.426599 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 21 05:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:42:05.433165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb55937-298a-4871-8fd7-fdd39df5a4b6 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:42:05.433992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb55937-298a-4871-8fd7-fdd39df5a4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:42:05.433992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb55937-298a-4871-8fd7-fdd39df5a4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:42:05.454265 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-2bb55937-298a-4871-8fd7-fdd39df5a4b6 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:42:05.455482 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-2bb55937-298a-4871-8fd7-fdd39df5a4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:42:05.464689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 21 05:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:42:05.766935 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841e157-23e0-49ed-ba84-1b48925bae9c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:42:05.769248 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-7841e157-23e0-49ed-ba84-1b48925bae9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:42:05.784129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841e157-23e0-49ed-ba84-1b48925bae9c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:42:05.788696 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed06b5-dadd-4c63-8a21-99c6e3f741b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:42:05.789339 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed06b5-dadd-4c63-8a21-99c6e3f741b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:42:05.789621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed06b5-dadd-4c63-8a21-99c6e3f741b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:42:05.789897 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed06b5-dadd-4c63-8a21-99c6e3f741b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:42:05.799544 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed06b5-dadd-4c63-8a21-99c6e3f741b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:42:05.800983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed06b5-dadd-4c63-8a21-99c6e3f741b0 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:42:05.810642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 768/1539] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:42:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:42:15.449221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=11 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:42:15.806766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=8 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:34.469470 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36925016-da2f-4066-8c53-a92cabe8e2b7 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:34.470173 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36925016-da2f-4066-8c53-a92cabe8e2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:34.470173 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36925016-da2f-4066-8c53-a92cabe8e2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:34.508388 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-36925016-da2f-4066-8c53-a92cabe8e2b7 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:34.509565 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-36925016-da2f-4066-8c53-a92cabe8e2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:34.524227 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 772/1540] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:43:34.533121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16fe75-d2fb-4603-b22a-0e109bcb2284 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:34.533747 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16fe75-d2fb-4603-b22a-0e109bcb2284 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:34.533747 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16fe75-d2fb-4603-b22a-0e109bcb2284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:34.557950 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-be16fe75-d2fb-4603-b22a-0e109bcb2284 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:34.559195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-be16fe75-d2fb-4603-b22a-0e109bcb2284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:34.571075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 769/1541] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:34.575293 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16fe75-d2fb-4603-b22a-0e109bcb2284 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:34.575605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d84783-cfe3-44cb-a4f2-a220ab41cfa4 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:34.575896 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d84783-cfe3-44cb-a4f2-a220ab41cfa4 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:34.576177 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d84783-cfe3-44cb-a4f2-a220ab41cfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:34.585498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16fe75-d2fb-4603-b22a-0e109bcb2284 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:34.594590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23a4aa-27a2-4905-8841-390ddaa9392f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:34.595297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23a4aa-27a2-4905-8841-390ddaa9392f admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:34.595581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23a4aa-27a2-4905-8841-390ddaa9392f admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:34.595947 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23a4aa-27a2-4905-8841-390ddaa9392f admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:34.596603 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23a4aa-27a2-4905-8841-390ddaa9392f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:43:34.596906 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23a4aa-27a2-4905-8841-390ddaa9392f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:34.597928 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: WARNING py.warnings [None req-af23a4aa-27a2-4905-8841-390ddaa9392f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 21 05:43:34.597928 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: warnings.warn(msg) Jul 21 05:43:34.597928 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: Jul 21 05:43:34.598811 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23a4aa-27a2-4905-8841-390ddaa9392f admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:34.620035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 770/1542] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:43:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:43:34.620269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-66d84783-cfe3-44cb-a4f2-a220ab41cfa4 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:34.621569 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-66d84783-cfe3-44cb-a4f2-a220ab41cfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:34.622613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23a4aa-27a2-4905-8841-390ddaa9392f admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:34.636948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23a4aa-27a2-4905-8841-390ddaa9392f admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:34.640905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 773/1543] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:34.642480 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966da49-f87e-4e6d-9ab3-da5d0e9153fc None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:34.642769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966da49-f87e-4e6d-9ab3-da5d0e9153fc None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:34.643062 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966da49-f87e-4e6d-9ab3-da5d0e9153fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:34.654440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d267e9f2-7ac6-41d3-85ab-266425bccf48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:34.654440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267e9f2-7ac6-41d3-85ab-266425bccf48 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:34.654440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267e9f2-7ac6-41d3-85ab-266425bccf48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:34.654818 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d267e9f2-7ac6-41d3-85ab-266425bccf48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:34.656569 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267e9f2-7ac6-41d3-85ab-266425bccf48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:34.658191 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d267e9f2-7ac6-41d3-85ab-266425bccf48 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:34.686288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-f966da49-f87e-4e6d-9ab3-da5d0e9153fc None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:34.686288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f966da49-f87e-4e6d-9ab3-da5d0e9153fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:34.709215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 774/1544] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:34.710712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909acef6-9af8-430d-bcb9-e3fb827756e4 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:34.711001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909acef6-9af8-430d-bcb9-e3fb827756e4 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:34.711001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909acef6-9af8-430d-bcb9-e3fb827756e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:34.754363 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-909acef6-9af8-430d-bcb9-e3fb827756e4 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:34.755643 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-909acef6-9af8-430d-bcb9-e3fb827756e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:34.756854 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 771/1545] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:43:34 2022] POST /identity/v3/projects => generated 389 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:43:34.758483 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024899ad-da71-485f-8bbd-e82fe729731a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:34.758933 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024899ad-da71-485f-8bbd-e82fe729731a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:34.759397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024899ad-da71-485f-8bbd-e82fe729731a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:34.792106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-024899ad-da71-485f-8bbd-e82fe729731a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:34.793617 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-024899ad-da71-485f-8bbd-e82fe729731a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:34.868770 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 772/1546] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:34.869984 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 775/1547] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:34.870708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2a878-8d0c-4a56-a13d-f6cc4106ea49 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:34.870839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909acef6-9af8-430d-bcb9-e3fb827756e4 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:34.871303 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2a878-8d0c-4a56-a13d-f6cc4106ea49 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:34.871787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2a878-8d0c-4a56-a13d-f6cc4106ea49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:34.884107 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909acef6-9af8-430d-bcb9-e3fb827756e4 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:34.888835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d80cb-2146-4d3e-9493-fe56b36ef0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:34.889514 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d80cb-2146-4d3e-9493-fe56b36ef0f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:34.889869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d80cb-2146-4d3e-9493-fe56b36ef0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:34.890160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d80cb-2146-4d3e-9493-fe56b36ef0f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:34.890283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-1ab2a878-8d0c-4a56-a13d-f6cc4106ea49 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:34.890853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d80cb-2146-4d3e-9493-fe56b36ef0f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:34.891425 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1ab2a878-8d0c-4a56-a13d-f6cc4106ea49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:34.892817 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d80cb-2146-4d3e-9493-fe56b36ef0f0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:34.899405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 773/1548] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:34.900437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a3751-0184-4fcf-8b44-f50b5cfe15c2 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:34.900725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a3751-0184-4fcf-8b44-f50b5cfe15c2 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:34.901002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a3751-0184-4fcf-8b44-f50b5cfe15c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:34.920379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-148a3751-0184-4fcf-8b44-f50b5cfe15c2 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:34.921328 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 776/1549] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:43:34 2022] POST /identity/v3/users => generated 295 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:34.923153 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-148a3751-0184-4fcf-8b44-f50b5cfe15c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:34.923272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8815a0-d7d2-47f1-8b7c-e692516f78b6 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:34.923272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8815a0-d7d2-47f1-8b7c-e692516f78b6 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:34.923455 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8815a0-d7d2-47f1-8b7c-e692516f78b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:34.932158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 774/1550] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:34.933278 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc981f39-7cf0-4c94-8d5f-638b6831c366 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:34.933559 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc981f39-7cf0-4c94-8d5f-638b6831c366 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:34.933884 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc981f39-7cf0-4c94-8d5f-638b6831c366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:34.943886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-be8815a0-d7d2-47f1-8b7c-e692516f78b6 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:34.945114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-be8815a0-d7d2-47f1-8b7c-e692516f78b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:34.951376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-fc981f39-7cf0-4c94-8d5f-638b6831c366 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:34.952664 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-fc981f39-7cf0-4c94-8d5f-638b6831c366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:34.953147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 777/1551] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:34.955947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8815a0-d7d2-47f1-8b7c-e692516f78b6 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:34.960820 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 775/1552] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:34.961610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc981f39-7cf0-4c94-8d5f-638b6831c366 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:34.966403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8815a0-d7d2-47f1-8b7c-e692516f78b6 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:34.970856 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc981f39-7cf0-4c94-8d5f-638b6831c366 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:34.971392 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03928770-4c69-460e-af5e-ff5a032a5c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:34.972056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03928770-4c69-460e-af5e-ff5a032a5c33 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:34.972360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03928770-4c69-460e-af5e-ff5a032a5c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:34.972629 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03928770-4c69-460e-af5e-ff5a032a5c33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:34.973384 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03928770-4c69-460e-af5e-ff5a032a5c33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:43:34.973706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03928770-4c69-460e-af5e-ff5a032a5c33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:34.975336 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03928770-4c69-460e-af5e-ff5a032a5c33 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:34.975526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b113313-89a9-4659-8437-d4b55f574b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:34.976151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b113313-89a9-4659-8437-d4b55f574b8e admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:34.976432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b113313-89a9-4659-8437-d4b55f574b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:34.976822 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b113313-89a9-4659-8437-d4b55f574b8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:34.977419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b113313-89a9-4659-8437-d4b55f574b8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:43:34.977720 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b113313-89a9-4659-8437-d4b55f574b8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:34.979656 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b113313-89a9-4659-8437-d4b55f574b8e admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:34.988154 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 778/1553] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:43:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:43:34.991976 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 776/1554] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:43:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:43:34.992553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b113313-89a9-4659-8437-d4b55f574b8e admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:34.996462 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9db49-8128-49ad-a8ec-bd2e9f2ce770 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:34.996754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9db49-8128-49ad-a8ec-bd2e9f2ce770 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:34.997036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9db49-8128-49ad-a8ec-bd2e9f2ce770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.007742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b113313-89a9-4659-8437-d4b55f574b8e admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.012618 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3353b-e7a6-4ba9-9427-68315e006c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.013275 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3353b-e7a6-4ba9-9427-68315e006c4c admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.014166 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3353b-e7a6-4ba9-9427-68315e006c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.014442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3353b-e7a6-4ba9-9427-68315e006c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.015331 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3353b-e7a6-4ba9-9427-68315e006c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.015531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-17f9db49-8128-49ad-a8ec-bd2e9f2ce770 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:35.016770 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-17f9db49-8128-49ad-a8ec-bd2e9f2ce770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:35.016954 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3353b-e7a6-4ba9-9427-68315e006c4c admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.024726 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 779/1555] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:35.026026 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4b9d5-10cb-4e72-aade-e6d77dd26ea0 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.026316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4b9d5-10cb-4e72-aade-e6d77dd26ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.026602 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4b9d5-10cb-4e72-aade-e6d77dd26ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.040252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 777/1556] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:35.040252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dab9a-a9b6-4236-9596-da6ce649d87b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.040252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dab9a-a9b6-4236-9596-da6ce649d87b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.040252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6dab9a-a9b6-4236-9596-da6ce649d87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.046779 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-6cd4b9d5-10cb-4e72-aade-e6d77dd26ea0 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:35.048024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-6cd4b9d5-10cb-4e72-aade-e6d77dd26ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:35.055990 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 780/1557] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:35.057144 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-6b6dab9a-a9b6-4236-9596-da6ce649d87b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:35.058373 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6b6dab9a-a9b6-4236-9596-da6ce649d87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:35.059830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bde6a9-2432-46d1-9c09-6a9c3a47d118 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.060426 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bde6a9-2432-46d1-9c09-6a9c3a47d118 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.060827 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bde6a9-2432-46d1-9c09-6a9c3a47d118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.066586 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 778/1558] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:35.067918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5ef22-d9d5-495f-8384-a8e2b80f2f4b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.068201 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5ef22-d9d5-495f-8384-a8e2b80f2f4b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.068472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5ef22-d9d5-495f-8384-a8e2b80f2f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.084774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-97bde6a9-2432-46d1-9c09-6a9c3a47d118 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:35.086641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-97bde6a9-2432-46d1-9c09-6a9c3a47d118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:35.087526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-29f5ef22-d9d5-495f-8384-a8e2b80f2f4b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:35.088898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-29f5ef22-d9d5-495f-8384-a8e2b80f2f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:35.095117 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 781/1559] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:35.096083 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bde6a9-2432-46d1-9c09-6a9c3a47d118 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.096879 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 779/1560] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:35.097481 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5ef22-d9d5-495f-8384-a8e2b80f2f4b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.107780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5ef22-d9d5-495f-8384-a8e2b80f2f4b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.107963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bde6a9-2432-46d1-9c09-6a9c3a47d118 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.112223 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ab8d4-286b-4155-92c9-4256438df52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.112429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fbe14-7f2d-4a68-850c-cf4bdcaa4ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.112862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ab8d4-286b-4155-92c9-4256438df52f admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.113078 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fbe14-7f2d-4a68-850c-cf4bdcaa4ad3 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.113179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ab8d4-286b-4155-92c9-4256438df52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.113368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fbe14-7f2d-4a68-850c-cf4bdcaa4ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.113481 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ab8d4-286b-4155-92c9-4256438df52f admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.113645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fbe14-7f2d-4a68-850c-cf4bdcaa4ad3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.114117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ab8d4-286b-4155-92c9-4256438df52f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.114333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fbe14-7f2d-4a68-850c-cf4bdcaa4ad3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.115400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: WARNING py.warnings [None req-454fbe14-7f2d-4a68-850c-cf4bdcaa4ad3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 05:43:35.115400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: warnings.warn(msg) Jul 21 05:43:35.115400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: Jul 21 05:43:35.116060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2ab8d4-286b-4155-92c9-4256438df52f admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.116580 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fbe14-7f2d-4a68-850c-cf4bdcaa4ad3 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.146353 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 780/1561] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:35.146353 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ab8d4-286b-4155-92c9-4256438df52f admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.146669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 782/1562] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:35.149007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fbe14-7f2d-4a68-850c-cf4bdcaa4ad3 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.179621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:35.210643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ab8d4-286b-4155-92c9-4256438df52f admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.213925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fbe14-7f2d-4a68-850c-cf4bdcaa4ad3 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.215406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1501c5f-fe53-4046-9c36-4cd6565cb66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.216081 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1501c5f-fe53-4046-9c36-4cd6565cb66e admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.216362 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1501c5f-fe53-4046-9c36-4cd6565cb66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.216636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1501c5f-fe53-4046-9c36-4cd6565cb66e admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.217367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1501c5f-fe53-4046-9c36-4cd6565cb66e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:43:35.217671 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1501c5f-fe53-4046-9c36-4cd6565cb66e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.218600 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75878e3b-9e97-4a0f-b37c-3862bf101b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.219298 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75878e3b-9e97-4a0f-b37c-3862bf101b6a admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.219417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1501c5f-fe53-4046-9c36-4cd6565cb66e admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.219586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75878e3b-9e97-4a0f-b37c-3862bf101b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.219861 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75878e3b-9e97-4a0f-b37c-3862bf101b6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.220587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75878e3b-9e97-4a0f-b37c-3862bf101b6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.222177 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75878e3b-9e97-4a0f-b37c-3862bf101b6a admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.224333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:35.238338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 781/1563] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:43:35.238338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a91cb3-99a2-4be8-9126-67053ae04f74 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.238338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a91cb3-99a2-4be8-9126-67053ae04f74 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.238338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a91cb3-99a2-4be8-9126-67053ae04f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.238778 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 783/1564] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:35.238778 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75878e3b-9e97-4a0f-b37c-3862bf101b6a admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.246866 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75878e3b-9e97-4a0f-b37c-3862bf101b6a admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.251171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-88a91cb3-99a2-4be8-9126-67053ae04f74 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:35.251557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8255d51-5ed9-4ab5-b154-a199439a5114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.252212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8255d51-5ed9-4ab5-b154-a199439a5114 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.252367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-88a91cb3-99a2-4be8-9126-67053ae04f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:35.252510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8255d51-5ed9-4ab5-b154-a199439a5114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.252758 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8255d51-5ed9-4ab5-b154-a199439a5114 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.253486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8255d51-5ed9-4ab5-b154-a199439a5114 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:43:35.253795 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8255d51-5ed9-4ab5-b154-a199439a5114 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.255447 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8255d51-5ed9-4ab5-b154-a199439a5114 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.260377 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 782/1565] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:35.261146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a91cb3-99a2-4be8-9126-67053ae04f74 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.270333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a91cb3-99a2-4be8-9126-67053ae04f74 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.271497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 784/1566] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:43:35.278555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8255d51-5ed9-4ab5-b154-a199439a5114 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.278681 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8787bb6-14b1-4f0b-bf1b-ea5c3fce2488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.278681 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8787bb6-14b1-4f0b-bf1b-ea5c3fce2488 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.278681 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8787bb6-14b1-4f0b-bf1b-ea5c3fce2488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.278681 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8787bb6-14b1-4f0b-bf1b-ea5c3fce2488 admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.279052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8787bb6-14b1-4f0b-bf1b-ea5c3fce2488 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:43:35.279052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8787bb6-14b1-4f0b-bf1b-ea5c3fce2488 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.279052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8787bb6-14b1-4f0b-bf1b-ea5c3fce2488 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.284576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8255d51-5ed9-4ab5-b154-a199439a5114 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.290056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ea00f-ae43-4679-a7d3-fd02b71378ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.290698 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ea00f-ae43-4679-a7d3-fd02b71378ef admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.290915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 783/1567] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:43:35.291054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ea00f-ae43-4679-a7d3-fd02b71378ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.291274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ea00f-ae43-4679-a7d3-fd02b71378ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.291757 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8787bb6-14b1-4f0b-bf1b-ea5c3fce2488 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.292116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ea00f-ae43-4679-a7d3-fd02b71378ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:43:35.292368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ea00f-ae43-4679-a7d3-fd02b71378ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.294348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ea00f-ae43-4679-a7d3-fd02b71378ef admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.305839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 785/1568] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:43:35.306897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ea00f-ae43-4679-a7d3-fd02b71378ef admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.307869 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8787bb6-14b1-4f0b-bf1b-ea5c3fce2488 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.312700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb1276-6cbd-4523-918d-69f4ef15a38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.314214 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb1276-6cbd-4523-918d-69f4ef15a38a admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.314501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb1276-6cbd-4523-918d-69f4ef15a38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.314785 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb1276-6cbd-4523-918d-69f4ef15a38a admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.315588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb1276-6cbd-4523-918d-69f4ef15a38a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.316616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: WARNING py.warnings [None req-84eb1276-6cbd-4523-918d-69f4ef15a38a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 05:43:35.316616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: warnings.warn(msg) Jul 21 05:43:35.316616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: Jul 21 05:43:35.316906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ea00f-ae43-4679-a7d3-fd02b71378ef admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.317828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb1276-6cbd-4523-918d-69f4ef15a38a admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.321299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1798912-706d-4af8-8b2b-95f6007ee2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.321941 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1798912-706d-4af8-8b2b-95f6007ee2a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.322213 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1798912-706d-4af8-8b2b-95f6007ee2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.322480 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1798912-706d-4af8-8b2b-95f6007ee2a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.323178 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1798912-706d-4af8-8b2b-95f6007ee2a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.325057 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1798912-706d-4af8-8b2b-95f6007ee2a1 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.337850 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 784/1569] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/users => generated 287 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:35.338783 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb1276-6cbd-4523-918d-69f4ef15a38a admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.350873 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 786/1570] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:35.351061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb1276-6cbd-4523-918d-69f4ef15a38a admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.351174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1798912-706d-4af8-8b2b-95f6007ee2a1 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.353110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661441cb-9d25-4baf-a20c-709f2522e679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.353983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661441cb-9d25-4baf-a20c-709f2522e679 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.354265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661441cb-9d25-4baf-a20c-709f2522e679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.354545 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661441cb-9d25-4baf-a20c-709f2522e679 admin admin] PATH_INFO: `/v3/projects/7b88b7408762493e825c86d05b5e62bc/users/0cbf797171144c139c72c516cea59bb9/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.358074 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1798912-706d-4af8-8b2b-95f6007ee2a1 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.362572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b7be2-bad0-4645-aa49-b36a5a481f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.363257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b7be2-bad0-4645-aa49-b36a5a481f52 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.363528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b7be2-bad0-4645-aa49-b36a5a481f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.363806 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b7be2-bad0-4645-aa49-b36a5a481f52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.364494 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b7be2-bad0-4645-aa49-b36a5a481f52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.366415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b7be2-bad0-4645-aa49-b36a5a481f52 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.369187 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661441cb-9d25-4baf-a20c-709f2522e679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b88b7408762493e825c86d05b5e62bc, user_id=0cbf797171144c139c72c516cea59bb9, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.371923 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661441cb-9d25-4baf-a20c-709f2522e679 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.386154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 785/1571] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:35 2022] PUT /identity/v3/projects/7b88b7408762493e825c86d05b5e62bc/users/0cbf797171144c139c72c516cea59bb9/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:35.386958 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661441cb-9d25-4baf-a20c-709f2522e679 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.398881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 787/1572] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/projects => generated 393 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:35.398881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b7be2-bad0-4645-aa49-b36a5a481f52 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.450206 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661441cb-9d25-4baf-a20c-709f2522e679 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.454891 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70846435-950f-444c-848d-301c921c949b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.455585 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70846435-950f-444c-848d-301c921c949b admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.455875 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70846435-950f-444c-848d-301c921c949b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.456156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70846435-950f-444c-848d-301c921c949b admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.456912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70846435-950f-444c-848d-301c921c949b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:43:35.457222 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70846435-950f-444c-848d-301c921c949b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.458359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b7be2-bad0-4645-aa49-b36a5a481f52 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.458844 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70846435-950f-444c-848d-301c921c949b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.463134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd864ba6-3c4f-4ce8-abc6-531de19677a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.463822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd864ba6-3c4f-4ce8-abc6-531de19677a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.464106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd864ba6-3c4f-4ce8-abc6-531de19677a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.464383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd864ba6-3c4f-4ce8-abc6-531de19677a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.465072 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd864ba6-3c4f-4ce8-abc6-531de19677a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.466971 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd864ba6-3c4f-4ce8-abc6-531de19677a6 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.470529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 786/1573] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:43:35.474645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70846435-950f-444c-848d-301c921c949b admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.485746 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70846435-950f-444c-848d-301c921c949b admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.491235 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08035793-7950-42d6-a516-eb48deb22135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.491934 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08035793-7950-42d6-a516-eb48deb22135 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.492330 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08035793-7950-42d6-a516-eb48deb22135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.492617 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08035793-7950-42d6-a516-eb48deb22135 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.493310 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08035793-7950-42d6-a516-eb48deb22135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.493605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 788/1574] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:35.494569 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd864ba6-3c4f-4ce8-abc6-531de19677a6 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.495368 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08035793-7950-42d6-a516-eb48deb22135 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.522530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 787/1575] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:35.524064 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08035793-7950-42d6-a516-eb48deb22135 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.557257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd864ba6-3c4f-4ce8-abc6-531de19677a6 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.562925 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:35.576801 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:35.588415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08035793-7950-42d6-a516-eb48deb22135 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.592985 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bab3f-b230-44a3-a9ca-b61759ce7bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.593683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bab3f-b230-44a3-a9ca-b61759ce7bed admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.593984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bab3f-b230-44a3-a9ca-b61759ce7bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.594245 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bab3f-b230-44a3-a9ca-b61759ce7bed admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.594983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bab3f-b230-44a3-a9ca-b61759ce7bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.596635 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bab3f-b230-44a3-a9ca-b61759ce7bed admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.606584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 788/1576] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:35.607439 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bab3f-b230-44a3-a9ca-b61759ce7bed admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.616472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b21840-acd6-4a95-8399-5f127449dd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.617061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bab3f-b230-44a3-a9ca-b61759ce7bed admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.617198 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b21840-acd6-4a95-8399-5f127449dd27 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.617450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b21840-acd6-4a95-8399-5f127449dd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.617716 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b21840-acd6-4a95-8399-5f127449dd27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.618442 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b21840-acd6-4a95-8399-5f127449dd27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.620437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b21840-acd6-4a95-8399-5f127449dd27 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.621536 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077bdf1-4c70-440d-b518-59bd4dc46195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.622174 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077bdf1-4c70-440d-b518-59bd4dc46195 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.622454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077bdf1-4c70-440d-b518-59bd4dc46195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.622726 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077bdf1-4c70-440d-b518-59bd4dc46195 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.623529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f077bdf1-4c70-440d-b518-59bd4dc46195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.625084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f077bdf1-4c70-440d-b518-59bd4dc46195 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.634914 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 789/1577] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:35.636155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dcf5f-5779-4640-b3c4-b4670da80e37 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.636437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dcf5f-5779-4640-b3c4-b4670da80e37 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.636707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dcf5f-5779-4640-b3c4-b4670da80e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.648866 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 789/1578] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:35.649706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b21840-acd6-4a95-8399-5f127449dd27 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.655002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-421dcf5f-5779-4640-b3c4-b4670da80e37 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:35.656481 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-421dcf5f-5779-4640-b3c4-b4670da80e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:35.714693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b21840-acd6-4a95-8399-5f127449dd27 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.717344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 790/1579] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:35.718035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421dcf5f-5779-4640-b3c4-b4670da80e37 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.722892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31efb492-f537-47c7-8779-7f908f3b6beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.723571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efb492-f537-47c7-8779-7f908f3b6beb admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.723860 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efb492-f537-47c7-8779-7f908f3b6beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.724162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efb492-f537-47c7-8779-7f908f3b6beb admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.724867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31efb492-f537-47c7-8779-7f908f3b6beb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.727037 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31efb492-f537-47c7-8779-7f908f3b6beb admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.727299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421dcf5f-5779-4640-b3c4-b4670da80e37 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.732007 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e9dc2-debc-4812-a183-7ca4ef4ef8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.732661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e9dc2-debc-4812-a183-7ca4ef4ef8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.732944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e9dc2-debc-4812-a183-7ca4ef4ef8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.733217 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e9dc2-debc-4812-a183-7ca4ef4ef8c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.733981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e9dc2-debc-4812-a183-7ca4ef4ef8c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.735927 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e9dc2-debc-4812-a183-7ca4ef4ef8c4 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.748182 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 790/1580] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:35.748946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31efb492-f537-47c7-8779-7f908f3b6beb admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.758635 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31efb492-f537-47c7-8779-7f908f3b6beb admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.763471 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba413d57-836d-4f1d-ae82-c59bfd629a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.767177 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 791/1581] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:35.767327 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba413d57-836d-4f1d-ae82-c59bfd629a9b admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.767327 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba413d57-836d-4f1d-ae82-c59bfd629a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.767506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e9dc2-debc-4812-a183-7ca4ef4ef8c4 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.767609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba413d57-836d-4f1d-ae82-c59bfd629a9b admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.767609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba413d57-836d-4f1d-ae82-c59bfd629a9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.767883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba413d57-836d-4f1d-ae82-c59bfd629a9b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.792966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 791/1582] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/users => generated 282 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:35.792966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba413d57-836d-4f1d-ae82-c59bfd629a9b admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.800951 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba413d57-836d-4f1d-ae82-c59bfd629a9b admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.830404 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e9dc2-debc-4812-a183-7ca4ef4ef8c4 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.835485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f9c32-b71d-4ccd-a5d7-c42d32bf5546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.836162 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f9c32-b71d-4ccd-a5d7-c42d32bf5546 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.836418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f9c32-b71d-4ccd-a5d7-c42d32bf5546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.836692 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f9c32-b71d-4ccd-a5d7-c42d32bf5546 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.837413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f9c32-b71d-4ccd-a5d7-c42d32bf5546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.839372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f9c32-b71d-4ccd-a5d7-c42d32bf5546 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.857421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e6afd-5635-40eb-abb4-0338cd2280ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.858221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e6afd-5635-40eb-abb4-0338cd2280ae admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.858512 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e6afd-5635-40eb-abb4-0338cd2280ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.858792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e6afd-5635-40eb-abb4-0338cd2280ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.859561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e6afd-5635-40eb-abb4-0338cd2280ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.862932 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 792/1583] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:35.862932 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f9c32-b71d-4ccd-a5d7-c42d32bf5546 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.863221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e6afd-5635-40eb-abb4-0338cd2280ae admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.870251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f9c32-b71d-4ccd-a5d7-c42d32bf5546 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.874689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 792/1584] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:35.874689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e6afd-5635-40eb-abb4-0338cd2280ae admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.878495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e39c2-07c2-4785-9d51-ed170456a1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.879332 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e39c2-07c2-4785-9d51-ed170456a1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.879621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e39c2-07c2-4785-9d51-ed170456a1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.879899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e39c2-07c2-4785-9d51-ed170456a1e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.880715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e39c2-07c2-4785-9d51-ed170456a1e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.882269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e6afd-5635-40eb-abb4-0338cd2280ae admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.882406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e39c2-07c2-4785-9d51-ed170456a1e3 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.886888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bd4f1-9a50-45f0-bb49-e7921cd1768d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.887595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bd4f1-9a50-45f0-bb49-e7921cd1768d admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.887884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bd4f1-9a50-45f0-bb49-e7921cd1768d admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.888158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bd4f1-9a50-45f0-bb49-e7921cd1768d admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.888841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0bd4f1-9a50-45f0-bb49-e7921cd1768d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.890784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0bd4f1-9a50-45f0-bb49-e7921cd1768d admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.895360 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 793/1585] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:35.895360 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e39c2-07c2-4785-9d51-ed170456a1e3 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.907649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:35.913802 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 793/1586] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:35.914823 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bd4f1-9a50-45f0-bb49-e7921cd1768d admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.918160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:35.927384 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bd4f1-9a50-45f0-bb49-e7921cd1768d admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.931952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ca18d-e5e6-41dc-8dd1-62f0fb5d75ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.932611 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ca18d-e5e6-41dc-8dd1-62f0fb5d75ac admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.932929 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ca18d-e5e6-41dc-8dd1-62f0fb5d75ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.933204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ca18d-e5e6-41dc-8dd1-62f0fb5d75ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.933927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ca18d-e5e6-41dc-8dd1-62f0fb5d75ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.935577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ca18d-e5e6-41dc-8dd1-62f0fb5d75ac admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.946353 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 794/1587] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:35.948045 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ca18d-e5e6-41dc-8dd1-62f0fb5d75ac admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.952269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e39c2-07c2-4785-9d51-ed170456a1e3 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.956014 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da1323-d172-4cfc-94a7-27f9e8c99cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.956667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da1323-d172-4cfc-94a7-27f9e8c99cfd service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.956949 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da1323-d172-4cfc-94a7-27f9e8c99cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.957239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da1323-d172-4cfc-94a7-27f9e8c99cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.958172 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ca18d-e5e6-41dc-8dd1-62f0fb5d75ac admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.962507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d2733-6234-42d3-b2f4-9a0d1f333f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.963195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d2733-6234-42d3-b2f4-9a0d1f333f18 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.963474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d2733-6234-42d3-b2f4-9a0d1f333f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.963850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d2733-6234-42d3-b2f4-9a0d1f333f18 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.964526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d2733-6234-42d3-b2f4-9a0d1f333f18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.966248 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da1323-d172-4cfc-94a7-27f9e8c99cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:35.966408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d2733-6234-42d3-b2f4-9a0d1f333f18 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.967714 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da1323-d172-4cfc-94a7-27f9e8c99cfd service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:35.977973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 794/1588] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:35.978846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da1323-d172-4cfc-94a7-27f9e8c99cfd service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.991865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 795/1589] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:43:35 2022] POST /identity/v3/users => generated 283 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:35.992068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da1323-d172-4cfc-94a7-27f9e8c99cfd service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:35.992168 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d2733-6234-42d3-b2f4-9a0d1f333f18 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:35.996336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdd235-0a5f-47ab-b5a3-c824d4878358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:35.997179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdd235-0a5f-47ab-b5a3-c824d4878358 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:35.997476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdd235-0a5f-47ab-b5a3-c824d4878358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:35.997839 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdd235-0a5f-47ab-b5a3-c824d4878358 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:35.998655 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdd235-0a5f-47ab-b5a3-c824d4878358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.000922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdd235-0a5f-47ab-b5a3-c824d4878358 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.002508 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d2733-6234-42d3-b2f4-9a0d1f333f18 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:36.008059 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4169e-397f-4737-8cfa-ddf63cb5fc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:36.008742 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4169e-397f-4737-8cfa-ddf63cb5fc16 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.008992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4169e-397f-4737-8cfa-ddf63cb5fc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.009270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4169e-397f-4737-8cfa-ddf63cb5fc16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.009983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4169e-397f-4737-8cfa-ddf63cb5fc16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.011628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4169e-397f-4737-8cfa-ddf63cb5fc16 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.013303 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 795/1590] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:36.013942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdd235-0a5f-47ab-b5a3-c824d4878358 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:36.021998 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 796/1591] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:36.022805 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4169e-397f-4737-8cfa-ddf63cb5fc16 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:36.028430 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdd235-0a5f-47ab-b5a3-c824d4878358 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:36.032344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4169e-397f-4737-8cfa-ddf63cb5fc16 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:36.034653 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5615ff1-19ef-4a8f-8d68-3290ab2eed58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:36.036156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5615ff1-19ef-4a8f-8d68-3290ab2eed58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.036533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5615ff1-19ef-4a8f-8d68-3290ab2eed58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.036908 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5615ff1-19ef-4a8f-8d68-3290ab2eed58 admin admin] PATH_INFO: `/v3/projects/19e2fcfba7b64bb5812c82d0944f4adc/users/7d36589cc19944f7ad37c3f6fd709b08/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.037242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f5a2f-46e7-4083-a357-75a1f3f187a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:36.038120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f5a2f-46e7-4083-a357-75a1f3f187a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.038398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f5a2f-46e7-4083-a357-75a1f3f187a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.038669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f5a2f-46e7-4083-a357-75a1f3f187a0 admin admin] PATH_INFO: `/v3/projects/5a36ae0fd7b648158ce3fbe24d474ad7/users/d01f138c8f0d4940af6737d2576a7326/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.058433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f5a2f-46e7-4083-a357-75a1f3f187a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a36ae0fd7b648158ce3fbe24d474ad7, user_id=d01f138c8f0d4940af6737d2576a7326, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.058953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5615ff1-19ef-4a8f-8d68-3290ab2eed58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19e2fcfba7b64bb5812c82d0944f4adc, user_id=7d36589cc19944f7ad37c3f6fd709b08, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.059979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: WARNING py.warnings [None req-192f5a2f-46e7-4083-a357-75a1f3f187a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 05:43:36.059979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: warnings.warn(msg) Jul 21 05:43:36.059979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: Jul 21 05:43:36.062811 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f5a2f-46e7-4083-a357-75a1f3f187a0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.065209 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5615ff1-19ef-4a8f-8d68-3290ab2eed58 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.082375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 797/1592] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:36 2022] PUT /identity/v3/projects/5a36ae0fd7b648158ce3fbe24d474ad7/users/d01f138c8f0d4940af6737d2576a7326/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:36.085576 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 796/1593] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:36 2022] PUT /identity/v3/projects/19e2fcfba7b64bb5812c82d0944f4adc/users/7d36589cc19944f7ad37c3f6fd709b08/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:36.097098 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5615ff1-19ef-4a8f-8d68-3290ab2eed58 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:36.097310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f5a2f-46e7-4083-a357-75a1f3f187a0 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:36.162670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5615ff1-19ef-4a8f-8d68-3290ab2eed58 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:36.168339 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f5a2f-46e7-4083-a357-75a1f3f187a0 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:36.169335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18808f-d2f3-4469-b561-b1d6f99731fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:36.170452 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18808f-d2f3-4469-b561-b1d6f99731fa admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.171089 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18808f-d2f3-4469-b561-b1d6f99731fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.171470 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18808f-d2f3-4469-b561-b1d6f99731fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.174782 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18808f-d2f3-4469-b561-b1d6f99731fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.174782 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18808f-d2f3-4469-b561-b1d6f99731fa admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.181618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99996943-4109-43ed-8fe9-6851b6dc395d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:36.182480 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99996943-4109-43ed-8fe9-6851b6dc395d admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.182625 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99996943-4109-43ed-8fe9-6851b6dc395d admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.182979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99996943-4109-43ed-8fe9-6851b6dc395d admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.183960 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99996943-4109-43ed-8fe9-6851b6dc395d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.185842 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99996943-4109-43ed-8fe9-6851b6dc395d admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.187608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 797/1594] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:36.189164 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18808f-d2f3-4469-b561-b1d6f99731fa admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:36.198467 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 798/1595] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:36.201115 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18808f-d2f3-4469-b561-b1d6f99731fa admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:36.202615 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99996943-4109-43ed-8fe9-6851b6dc395d admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:36.206872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252ce5c-9676-4844-ad77-c164d838065b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:36.207590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252ce5c-9676-4844-ad77-c164d838065b admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.207884 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252ce5c-9676-4844-ad77-c164d838065b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.208158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252ce5c-9676-4844-ad77-c164d838065b admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.209041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252ce5c-9676-4844-ad77-c164d838065b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.210509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252ce5c-9676-4844-ad77-c164d838065b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.213092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99996943-4109-43ed-8fe9-6851b6dc395d admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:36.217950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f48e6-02cf-438f-be55-b03df26d8eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:36.218622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f48e6-02cf-438f-be55-b03df26d8eba admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.218908 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f48e6-02cf-438f-be55-b03df26d8eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.219202 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f48e6-02cf-438f-be55-b03df26d8eba admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.219939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f48e6-02cf-438f-be55-b03df26d8eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.221732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 798/1596] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:36.221854 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f48e6-02cf-438f-be55-b03df26d8eba admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.226503 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252ce5c-9676-4844-ad77-c164d838065b admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:36.232081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 799/1597] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:36.233262 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5cc8a-40f3-4b11-a815-ab48085346c8 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.233678 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5cc8a-40f3-4b11-a815-ab48085346c8 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.233953 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5cc8a-40f3-4b11-a815-ab48085346c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.240591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252ce5c-9676-4844-ad77-c164d838065b admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:36.246519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4679e87-a0a0-4cce-bd4d-274fdbeaf49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:36.246961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4679e87-a0a0-4cce-bd4d-274fdbeaf49b admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.247276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4679e87-a0a0-4cce-bd4d-274fdbeaf49b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.247550 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4679e87-a0a0-4cce-bd4d-274fdbeaf49b admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.248354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4679e87-a0a0-4cce-bd4d-274fdbeaf49b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.250191 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4679e87-a0a0-4cce-bd4d-274fdbeaf49b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.252330 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-fed5cc8a-40f3-4b11-a815-ab48085346c8 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:36.253730 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-fed5cc8a-40f3-4b11-a815-ab48085346c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:36.261820 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 799/1598] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:36.263513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ac260-76f5-4d26-848c-dbb5d6c8178c None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.264012 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ac260-76f5-4d26-848c-dbb5d6c8178c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.264357 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ac260-76f5-4d26-848c-dbb5d6c8178c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.268421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 800/1599] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:36.268421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5cc8a-40f3-4b11-a815-ab48085346c8 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:36.276292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5cc8a-40f3-4b11-a815-ab48085346c8 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:36.281296 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e511d2b-6070-4e8d-893a-68b77104bc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:36.282108 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e511d2b-6070-4e8d-893a-68b77104bc9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.282391 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e511d2b-6070-4e8d-893a-68b77104bc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.282664 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e511d2b-6070-4e8d-893a-68b77104bc9e admin admin] PATH_INFO: `/v3/projects/da2259120cca423686fd6e54d0ad2257/users/563f1367093e493bae87a21374a3c2f3/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.300924 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e511d2b-6070-4e8d-893a-68b77104bc9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=da2259120cca423686fd6e54d0ad2257, user_id=563f1367093e493bae87a21374a3c2f3, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.303130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-dd4ac260-76f5-4d26-848c-dbb5d6c8178c None None] MFA Rules not processed for user `7d36589cc19944f7ad37c3f6fd709b08`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:36.303656 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e511d2b-6070-4e8d-893a-68b77104bc9e admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.304697 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-dd4ac260-76f5-4d26-848c-dbb5d6c8178c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:36.319715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 801/1600] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:36 2022] PUT /identity/v3/projects/da2259120cca423686fd6e54d0ad2257/users/563f1367093e493bae87a21374a3c2f3/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:36.320932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e511d2b-6070-4e8d-893a-68b77104bc9e admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:36.384551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:36.394124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:36.412573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e511d2b-6070-4e8d-893a-68b77104bc9e admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:36.417447 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 800/1601] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:36.417447 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ac260-76f5-4d26-848c-dbb5d6c8178c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:36.422993 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d66b2-2f69-4dd8-bfd8-14bccbc235e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:36.424166 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d66b2-2f69-4dd8-bfd8-14bccbc235e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.424605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d66b2-2f69-4dd8-bfd8-14bccbc235e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.425090 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d66b2-2f69-4dd8-bfd8-14bccbc235e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.425890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d66b2-2f69-4dd8-bfd8-14bccbc235e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.428304 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d66b2-2f69-4dd8-bfd8-14bccbc235e7 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.433358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ac260-76f5-4d26-848c-dbb5d6c8178c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:36.440233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c5fbc-0dac-4371-b39c-f09a0dce3c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:36.441424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c5fbc-0dac-4371-b39c-f09a0dce3c0a admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.441905 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c5fbc-0dac-4371-b39c-f09a0dce3c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.442314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c5fbc-0dac-4371-b39c-f09a0dce3c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.443266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c5fbc-0dac-4371-b39c-f09a0dce3c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.447498 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 802/1602] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:36.448533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c5fbc-0dac-4371-b39c-f09a0dce3c0a admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.448678 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d66b2-2f69-4dd8-bfd8-14bccbc235e7 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:36.460360 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 801/1603] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:36.460630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d66b2-2f69-4dd8-bfd8-14bccbc235e7 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:36.461170 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c5fbc-0dac-4371-b39c-f09a0dce3c0a admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:36.466734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211401ea-a8d8-4ab6-94fb-8f961e6d80c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:36.467943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211401ea-a8d8-4ab6-94fb-8f961e6d80c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.468139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211401ea-a8d8-4ab6-94fb-8f961e6d80c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.468387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211401ea-a8d8-4ab6-94fb-8f961e6d80c2 admin admin] PATH_INFO: `/v3/projects/294cef7518074b2f8554bd35829af503/users/c909ca17e90a4196bc21636a9f101b99/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.472144 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c5fbc-0dac-4371-b39c-f09a0dce3c0a admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:36.477378 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f06303-1c7b-4773-9f50-503b74be6b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:36.478276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f06303-1c7b-4773-9f50-503b74be6b02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.478565 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f06303-1c7b-4773-9f50-503b74be6b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.478969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f06303-1c7b-4773-9f50-503b74be6b02 admin admin] PATH_INFO: `/v3/projects/56e1c60a9af04d57a84e220a9add1553/users/7e08156ce92b49df803ea339b6b4c786/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.487156 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211401ea-a8d8-4ab6-94fb-8f961e6d80c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=294cef7518074b2f8554bd35829af503, user_id=c909ca17e90a4196bc21636a9f101b99, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.490002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211401ea-a8d8-4ab6-94fb-8f961e6d80c2 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.497169 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f06303-1c7b-4773-9f50-503b74be6b02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56e1c60a9af04d57a84e220a9add1553, user_id=7e08156ce92b49df803ea339b6b4c786, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.500895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f06303-1c7b-4773-9f50-503b74be6b02 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.506750 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 803/1604] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:36 2022] PUT /identity/v3/projects/294cef7518074b2f8554bd35829af503/users/c909ca17e90a4196bc21636a9f101b99/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:36.507537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211401ea-a8d8-4ab6-94fb-8f961e6d80c2 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:36.509839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-211401ea-a8d8-4ab6-94fb-8f961e6d80c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:36.531059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 802/1605] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:36 2022] PUT /identity/v3/projects/56e1c60a9af04d57a84e220a9add1553/users/7e08156ce92b49df803ea339b6b4c786/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:36.531059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b371fa1-6934-4637-a0da-27ff08f098e2 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.531059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b371fa1-6934-4637-a0da-27ff08f098e2 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.531059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b371fa1-6934-4637-a0da-27ff08f098e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.545432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-8b371fa1-6934-4637-a0da-27ff08f098e2 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:36.546673 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8b371fa1-6934-4637-a0da-27ff08f098e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:36.577754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211401ea-a8d8-4ab6-94fb-8f961e6d80c2 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:36.611646 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 803/1606] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:36.612411 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b371fa1-6934-4637-a0da-27ff08f098e2 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:36.622917 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b371fa1-6934-4637-a0da-27ff08f098e2 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:36.628435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6f5e1-fc55-44b6-a58b-d595ca31a662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:36.629252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6f5e1-fc55-44b6-a58b-d595ca31a662 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.629651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6f5e1-fc55-44b6-a58b-d595ca31a662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.629929 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6f5e1-fc55-44b6-a58b-d595ca31a662 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.630705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6f5e1-fc55-44b6-a58b-d595ca31a662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.633241 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6f5e1-fc55-44b6-a58b-d595ca31a662 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.637727 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d88518-e00e-4b1f-bfab-c11ef08bef20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:36.638821 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d88518-e00e-4b1f-bfab-c11ef08bef20 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.639198 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d88518-e00e-4b1f-bfab-c11ef08bef20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.640441 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d88518-e00e-4b1f-bfab-c11ef08bef20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.665984 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d88518-e00e-4b1f-bfab-c11ef08bef20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.669588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 804/1607] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:43:36 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:36.669755 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d88518-e00e-4b1f-bfab-c11ef08bef20 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.673929 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6f5e1-fc55-44b6-a58b-d595ca31a662 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:36.681936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 804/1608] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:36.692625 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d88518-e00e-4b1f-bfab-c11ef08bef20 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:36.707265 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d88518-e00e-4b1f-bfab-c11ef08bef20 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:36.825637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6f5e1-fc55-44b6-a58b-d595ca31a662 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:36.835447 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a90c8-3d66-4a90-a0cb-6498a2a59836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:36.837033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a90c8-3d66-4a90-a0cb-6498a2a59836 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.837033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a90c8-3d66-4a90-a0cb-6498a2a59836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.837318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a90c8-3d66-4a90-a0cb-6498a2a59836 admin admin] PATH_INFO: `/v3/projects/80c9f6c2ae394669be9a3a7efbb400fe/users/762c9249de0f45d88242e9f4e27ab246/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.843076 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccf5c2-620f-4086-b1b0-2be2a5b0b654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:36.844221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccf5c2-620f-4086-b1b0-2be2a5b0b654 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.844662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccf5c2-620f-4086-b1b0-2be2a5b0b654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.845079 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccf5c2-620f-4086-b1b0-2be2a5b0b654 admin admin] PATH_INFO: `/v3/projects/28238eceb52e42d985df53b2ba8a42df/users/45e9f4e991814209a322a78136e9bf19/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.859864 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a90c8-3d66-4a90-a0cb-6498a2a59836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80c9f6c2ae394669be9a3a7efbb400fe, user_id=762c9249de0f45d88242e9f4e27ab246, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.863966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a90c8-3d66-4a90-a0cb-6498a2a59836 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.883608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccf5c2-620f-4086-b1b0-2be2a5b0b654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28238eceb52e42d985df53b2ba8a42df, user_id=45e9f4e991814209a322a78136e9bf19, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:36.886171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 805/1609] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:36 2022] PUT /identity/v3/projects/80c9f6c2ae394669be9a3a7efbb400fe/users/762c9249de0f45d88242e9f4e27ab246/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:36.886333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccf5c2-620f-4086-b1b0-2be2a5b0b654 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:36.900171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeae36-4478-4fdd-bbd2-aaf427c331a1 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.900504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeae36-4478-4fdd-bbd2-aaf427c331a1 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.901092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbeae36-4478-4fdd-bbd2-aaf427c331a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.916303 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 805/1610] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:36 2022] PUT /identity/v3/projects/28238eceb52e42d985df53b2ba8a42df/users/45e9f4e991814209a322a78136e9bf19/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:36.918129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e389b9-3c46-417e-9b6c-55513b343b29 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:36.918522 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e389b9-3c46-417e-9b6c-55513b343b29 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:36.919153 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e389b9-3c46-417e-9b6c-55513b343b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:36.926628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:36.955255 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-0cbeae36-4478-4fdd-bbd2-aaf427c331a1 None None] MFA Rules not processed for user `c909ca17e90a4196bc21636a9f101b99`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:36.957654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0cbeae36-4478-4fdd-bbd2-aaf427c331a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:36.974630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-56e389b9-3c46-417e-9b6c-55513b343b29 None None] MFA Rules not processed for user `7e08156ce92b49df803ea339b6b4c786`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:36.976066 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-56e389b9-3c46-417e-9b6c-55513b343b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:36.985630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:37.046427 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 806/1611] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:36 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:37.046427 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbeae36-4478-4fdd-bbd2-aaf427c331a1 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:37.110999 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbeae36-4478-4fdd-bbd2-aaf427c331a1 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:37.115249 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6770244-4d86-47f5-a28a-d0864c8c55a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:37.115982 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6770244-4d86-47f5-a28a-d0864c8c55a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:37.116381 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6770244-4d86-47f5-a28a-d0864c8c55a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:37.116664 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6770244-4d86-47f5-a28a-d0864c8c55a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:37.124970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 806/1612] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:37.124970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e389b9-3c46-417e-9b6c-55513b343b29 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:37.226007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6770244-4d86-47f5-a28a-d0864c8c55a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:37.226243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e389b9-3c46-417e-9b6c-55513b343b29 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:37.227716 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6770244-4d86-47f5-a28a-d0864c8c55a9 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:37.231973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1db971-e3c8-4155-8139-9bd44e2800fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:37.232806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1db971-e3c8-4155-8139-9bd44e2800fe admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:37.233236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1db971-e3c8-4155-8139-9bd44e2800fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:37.233722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1db971-e3c8-4155-8139-9bd44e2800fe admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:37.234616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1db971-e3c8-4155-8139-9bd44e2800fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:37.237385 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1db971-e3c8-4155-8139-9bd44e2800fe admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:37.240376 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 807/1613] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:37.241754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552efd92-d266-43b1-a322-4a455b5c9cbd None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:37.242049 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552efd92-d266-43b1-a322-4a455b5c9cbd None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:37.242329 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552efd92-d266-43b1-a322-4a455b5c9cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:37.271051 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-552efd92-d266-43b1-a322-4a455b5c9cbd None None] MFA Rules not processed for user `762c9249de0f45d88242e9f4e27ab246`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:37.272964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-552efd92-d266-43b1-a322-4a455b5c9cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:37.275998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 807/1614] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:43:37 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:37.277884 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96476434-2dec-4084-be18-8d43c8f15a19 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:37.278282 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96476434-2dec-4084-be18-8d43c8f15a19 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:37.278684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96476434-2dec-4084-be18-8d43c8f15a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:37.313807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:37.332210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-96476434-2dec-4084-be18-8d43c8f15a19 None None] MFA Rules not processed for user `45e9f4e991814209a322a78136e9bf19`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:37.333707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-96476434-2dec-4084-be18-8d43c8f15a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:37.357614 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 808/1615] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:37.357949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552efd92-d266-43b1-a322-4a455b5c9cbd None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:37.369769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552efd92-d266-43b1-a322-4a455b5c9cbd None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:37.378951 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb4437-3237-47b7-8485-c266ac73fd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:37.379783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb4437-3237-47b7-8485-c266ac73fd34 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:37.380001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb4437-3237-47b7-8485-c266ac73fd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:37.380232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb4437-3237-47b7-8485-c266ac73fd34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:37.380936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb4437-3237-47b7-8485-c266ac73fd34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:37.384033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb4437-3237-47b7-8485-c266ac73fd34 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:37.421306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 809/1616] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:43:37 2022] POST /identity/v3/projects => generated 363 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:37.423379 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e0c45-3eab-45a7-8dd8-419f879ee48b None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:37.423706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e0c45-3eab-45a7-8dd8-419f879ee48b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:37.424127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e0c45-3eab-45a7-8dd8-419f879ee48b None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:37.432700 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 810/1617] 10.222.0.44 () {54 vars in 962 bytes} [Thu Jul 21 05:43:37 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:43:37.433763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e0c45-3eab-45a7-8dd8-419f879ee48b None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:37.448251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 808/1618] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:37.448251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96476434-2dec-4084-be18-8d43c8f15a19 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:37.483412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96476434-2dec-4084-be18-8d43c8f15a19 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:37.546579 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:37.551845 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e0c45-3eab-45a7-8dd8-419f879ee48b None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:37.559394 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d513e-d664-44b1-89cb-303b9158f25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:37.560099 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d513e-d664-44b1-89cb-303b9158f25f admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:37.560382 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d513e-d664-44b1-89cb-303b9158f25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:37.560704 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d513e-d664-44b1-89cb-303b9158f25f admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:37.563718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d513e-d664-44b1-89cb-303b9158f25f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:37.570036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d513e-d664-44b1-89cb-303b9158f25f admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:37.592074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1658081-bff5-4a90-aa79-ad1bd3e26b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:37.592674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 811/1619] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:37.593106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1658081-bff5-4a90-aa79-ad1bd3e26b17 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:37.593487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1658081-bff5-4a90-aa79-ad1bd3e26b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:37.594031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1658081-bff5-4a90-aa79-ad1bd3e26b17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:37.595058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1658081-bff5-4a90-aa79-ad1bd3e26b17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:37.597779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1658081-bff5-4a90-aa79-ad1bd3e26b17 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:37.598032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d513e-d664-44b1-89cb-303b9158f25f admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:37.625510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d513e-d664-44b1-89cb-303b9158f25f admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:37.630940 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f4a6b-67a3-4527-9d6d-dad3b16cacef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:37.631703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f4a6b-67a3-4527-9d6d-dad3b16cacef admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:37.631984 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f4a6b-67a3-4527-9d6d-dad3b16cacef admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:37.632275 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f4a6b-67a3-4527-9d6d-dad3b16cacef admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:37.632971 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f4a6b-67a3-4527-9d6d-dad3b16cacef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:37.634940 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f4a6b-67a3-4527-9d6d-dad3b16cacef admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:37.639204 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 809/1620] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:43:37 2022] POST /identity/v3/projects => generated 365 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:37.639204 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf328e-aa92-4681-80a9-5dddf48cae13 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:37.639204 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf328e-aa92-4681-80a9-5dddf48cae13 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:37.639204 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caf328e-aa92-4681-80a9-5dddf48cae13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:37.669212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 812/1621] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:43:37 2022] POST /identity/v3/users => generated 288 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:37.670880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b690d31-4374-4439-b65a-c1761557d4de None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:37.671264 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b690d31-4374-4439-b65a-c1761557d4de None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:37.672457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b690d31-4374-4439-b65a-c1761557d4de None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:37.673797 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 813/1622] 10.222.0.44 () {54 vars in 962 bytes} [Thu Jul 21 05:43:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:43:37.674403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b690d31-4374-4439-b65a-c1761557d4de None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:37.696745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-4caf328e-aa92-4681-80a9-5dddf48cae13 None None] MFA Rules not processed for user `928d014648cd41d7b58e7e01845eb748`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:37.698887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4caf328e-aa92-4681-80a9-5dddf48cae13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:37.750909 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b690d31-4374-4439-b65a-c1761557d4de None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:37.756127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304f410-5824-4488-9342-2cb066751170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:37.757307 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304f410-5824-4488-9342-2cb066751170 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:37.757587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304f410-5824-4488-9342-2cb066751170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:37.758115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304f410-5824-4488-9342-2cb066751170 admin admin] PATH_INFO: `/v3/projects/f216dbbb2bce4ef9be103afb5d2d5f66/users/19e7f57412964d75859506651a137d47/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:37.775921 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304f410-5824-4488-9342-2cb066751170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f216dbbb2bce4ef9be103afb5d2d5f66, user_id=19e7f57412964d75859506651a137d47, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:37.778621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304f410-5824-4488-9342-2cb066751170 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:37.782990 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 810/1623] 10.222.0.44 () {58 vars in 1059 bytes} [Thu Jul 21 05:43:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:37.784298 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf328e-aa92-4681-80a9-5dddf48cae13 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:37.798665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caf328e-aa92-4681-80a9-5dddf48cae13 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:37.798869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 814/1624] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:37 2022] PUT /identity/v3/projects/f216dbbb2bce4ef9be103afb5d2d5f66/users/19e7f57412964d75859506651a137d47/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:37.801871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304f410-5824-4488-9342-2cb066751170 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:37.822006 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:37.951544 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c5e27-4195-42a3-856d-0b53d7315d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:37.951544 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c5e27-4195-42a3-856d-0b53d7315d30 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:37.951544 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c5e27-4195-42a3-856d-0b53d7315d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:37.951544 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c5e27-4195-42a3-856d-0b53d7315d30 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:37.953055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c5e27-4195-42a3-856d-0b53d7315d30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:37.953055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c5e27-4195-42a3-856d-0b53d7315d30 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:37.953407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304f410-5824-4488-9342-2cb066751170 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:37.953407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1144f5-09d0-4ed4-b5a8-54ba71c22fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:37.953407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1144f5-09d0-4ed4-b5a8-54ba71c22fec admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:37.953407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1144f5-09d0-4ed4-b5a8-54ba71c22fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:37.953882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1144f5-09d0-4ed4-b5a8-54ba71c22fec admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:37.953882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1144f5-09d0-4ed4-b5a8-54ba71c22fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:37.953882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1144f5-09d0-4ed4-b5a8-54ba71c22fec admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:37.954146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 811/1625] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:43:37 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:37.954146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8dcc9-7ccb-4437-91c8-732321d39921 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:37.954146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8dcc9-7ccb-4437-91c8-732321d39921 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:37.954146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8dcc9-7ccb-4437-91c8-732321d39921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:37.954500 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 815/1626] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:37.954500 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1144f5-09d0-4ed4-b5a8-54ba71c22fec admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:37.959351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-dde8dcc9-7ccb-4437-91c8-732321d39921 None None] MFA Rules not processed for user `928d014648cd41d7b58e7e01845eb748`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:37.961973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-dde8dcc9-7ccb-4437-91c8-732321d39921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:37.986323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:38.048006 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1144f5-09d0-4ed4-b5a8-54ba71c22fec admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:38.054588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88077ea0-ef65-4239-a31c-6094a2464d39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:38.055456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88077ea0-ef65-4239-a31c-6094a2464d39 service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:38.055940 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88077ea0-ef65-4239-a31c-6094a2464d39 service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:38.056283 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88077ea0-ef65-4239-a31c-6094a2464d39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:38.068062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 812/1627] 10.222.0.44 () {58 vars in 1059 bytes} [Thu Jul 21 05:43:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:38.069105 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8dcc9-7ccb-4437-91c8-732321d39921 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:38.085952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8dcc9-7ccb-4437-91c8-732321d39921 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:38.092384 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89ca15-55f5-4abd-9c4e-813d9df3c267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:38.093398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89ca15-55f5-4abd-9c4e-813d9df3c267 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:38.093683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89ca15-55f5-4abd-9c4e-813d9df3c267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:38.093988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89ca15-55f5-4abd-9c4e-813d9df3c267 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:38.094770 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89ca15-55f5-4abd-9c4e-813d9df3c267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:38.097071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89ca15-55f5-4abd-9c4e-813d9df3c267 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:38.111561 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 813/1628] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:38.112570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89ca15-55f5-4abd-9c4e-813d9df3c267 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:38.122563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89ca15-55f5-4abd-9c4e-813d9df3c267 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:38.128618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88077ea0-ef65-4239-a31c-6094a2464d39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:38.129451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70469bbc-0f18-4e04-828e-ad0299e63176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:38.130124 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70469bbc-0f18-4e04-828e-ad0299e63176 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:38.130310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88077ea0-ef65-4239-a31c-6094a2464d39 service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:38.130434 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70469bbc-0f18-4e04-828e-ad0299e63176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:38.130778 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70469bbc-0f18-4e04-828e-ad0299e63176 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:38.131406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70469bbc-0f18-4e04-828e-ad0299e63176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:38.133111 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70469bbc-0f18-4e04-828e-ad0299e63176 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:38.142422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 816/1629] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:38.143320 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88077ea0-ef65-4239-a31c-6094a2464d39 service nova] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:38.144020 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 814/1630] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:38.144756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70469bbc-0f18-4e04-828e-ad0299e63176 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:38.159689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88077ea0-ef65-4239-a31c-6094a2464d39 service nova] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:38.164746 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70469bbc-0f18-4e04-828e-ad0299e63176 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:38.164946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813561d-26be-4859-8003-3c5f7e1a39e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:38.165665 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813561d-26be-4859-8003-3c5f7e1a39e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:38.165947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813561d-26be-4859-8003-3c5f7e1a39e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:38.166221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813561d-26be-4859-8003-3c5f7e1a39e0 admin admin] PATH_INFO: `/v3/projects/894f846ef7394eeebc695b698669cd56/users/3e8509e8358c462f89783dea99e00acc/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:38.172559 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085b817-50d1-4c63-84d1-f0391b53a8fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:38.173234 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085b817-50d1-4c63-84d1-f0391b53a8fd service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:38.173522 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085b817-50d1-4c63-84d1-f0391b53a8fd service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:38.173803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085b817-50d1-4c63-84d1-f0391b53a8fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:38.184699 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813561d-26be-4859-8003-3c5f7e1a39e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=894f846ef7394eeebc695b698669cd56, user_id=3e8509e8358c462f89783dea99e00acc, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:38.187713 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813561d-26be-4859-8003-3c5f7e1a39e0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:38.211724 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 817/1631] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:38 2022] PUT /identity/v3/projects/894f846ef7394eeebc695b698669cd56/users/3e8509e8358c462f89783dea99e00acc/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:38.211724 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813561d-26be-4859-8003-3c5f7e1a39e0 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:38.250393 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:38.268660 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085b817-50d1-4c63-84d1-f0391b53a8fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:38.270314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085b817-50d1-4c63-84d1-f0391b53a8fd service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:38.293553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 815/1632] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:38.294461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085b817-50d1-4c63-84d1-f0391b53a8fd service nova] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:38.310552 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085b817-50d1-4c63-84d1-f0391b53a8fd service nova] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:38.316222 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a864f-1e72-4511-b33f-bc9e306050d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:38.318162 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a864f-1e72-4511-b33f-bc9e306050d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:38.318644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a864f-1e72-4511-b33f-bc9e306050d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:38.319104 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a864f-1e72-4511-b33f-bc9e306050d8 admin admin] PATH_INFO: `/v3/projects/c19694f1c8f641359c4da57b70fffc90/users/0b60a90fc604451c848e908830c44afa/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:38.339553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813561d-26be-4859-8003-3c5f7e1a39e0 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:38.347617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ab947-4132-4539-8a5a-11746029e594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:38.348606 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ab947-4132-4539-8a5a-11746029e594 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:38.349041 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ab947-4132-4539-8a5a-11746029e594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:38.349320 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ab947-4132-4539-8a5a-11746029e594 admin admin] PATH_INFO: `/v3/projects/f216dbbb2bce4ef9be103afb5d2d5f66/users/19e7f57412964d75859506651a137d47/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:38.353309 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ab947-4132-4539-8a5a-11746029e594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f216dbbb2bce4ef9be103afb5d2d5f66, user_id=19e7f57412964d75859506651a137d47, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:38.357069 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ab947-4132-4539-8a5a-11746029e594 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:38.370933 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a864f-1e72-4511-b33f-bc9e306050d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c19694f1c8f641359c4da57b70fffc90, user_id=0b60a90fc604451c848e908830c44afa, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:38.373681 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a864f-1e72-4511-b33f-bc9e306050d8 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:38.386725 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 818/1633] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:38 2022] PUT /identity/v3/projects/f216dbbb2bce4ef9be103afb5d2d5f66/users/19e7f57412964d75859506651a137d47/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:38.388758 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ab947-4132-4539-8a5a-11746029e594 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:38.409547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 816/1634] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:38 2022] PUT /identity/v3/projects/c19694f1c8f641359c4da57b70fffc90/users/0b60a90fc604451c848e908830c44afa/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:38.411419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d77585-a1b0-40e7-bade-078c382e60fc None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:38.411875 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d77585-a1b0-40e7-bade-078c382e60fc None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:38.412314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d77585-a1b0-40e7-bade-078c382e60fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:38.474558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-58d77585-a1b0-40e7-bade-078c382e60fc None None] MFA Rules not processed for user `19e7f57412964d75859506651a137d47`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:38.475846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-58d77585-a1b0-40e7-bade-078c382e60fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:38.481224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ab947-4132-4539-8a5a-11746029e594 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:38.573277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606e25a-ef72-4636-857a-cd6d560d8b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:38.579223 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:38.589816 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606e25a-ef72-4636-857a-cd6d560d8b70 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:38.590101 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606e25a-ef72-4636-857a-cd6d560d8b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:38.590381 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606e25a-ef72-4636-857a-cd6d560d8b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:38.594788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606e25a-ef72-4636-857a-cd6d560d8b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:38.600792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606e25a-ef72-4636-857a-cd6d560d8b70 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:38.618136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 819/1635] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:38.618136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606e25a-ef72-4636-857a-cd6d560d8b70 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:38.639778 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 817/1636] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:38 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:38.643653 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926b944-f937-4efe-a4f9-4414cfb49571 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:38.643947 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926b944-f937-4efe-a4f9-4414cfb49571 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:38.644229 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926b944-f937-4efe-a4f9-4414cfb49571 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:38.653123 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606e25a-ef72-4636-857a-cd6d560d8b70 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:38.655493 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 818/1637] 10.222.0.44 () {54 vars in 912 bytes} [Thu Jul 21 05:43:38 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:43:38.656336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926b944-f937-4efe-a4f9-4414cfb49571 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:38.680208 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4100e-a1d3-4e27-855a-4eaa6ed3ce55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:38.680933 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4100e-a1d3-4e27-855a-4eaa6ed3ce55 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:38.681676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4100e-a1d3-4e27-855a-4eaa6ed3ce55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:38.681676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4100e-a1d3-4e27-855a-4eaa6ed3ce55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:38.682209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4100e-a1d3-4e27-855a-4eaa6ed3ce55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:38.689664 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4100e-a1d3-4e27-855a-4eaa6ed3ce55 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:38.705596 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926b944-f937-4efe-a4f9-4414cfb49571 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:38.719980 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 820/1638] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:38.722291 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4100e-a1d3-4e27-855a-4eaa6ed3ce55 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:38.726080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d14e1-a143-43dc-8b0e-2e25bf868c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:38.727002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d14e1-a143-43dc-8b0e-2e25bf868c6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:38.731787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d14e1-a143-43dc-8b0e-2e25bf868c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:38.732083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d14e1-a143-43dc-8b0e-2e25bf868c6b admin admin] PATH_INFO: `/v3/projects/894f846ef7394eeebc695b698669cd56/users/3e8509e8358c462f89783dea99e00acc/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:38.736051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d14e1-a143-43dc-8b0e-2e25bf868c6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=894f846ef7394eeebc695b698669cd56, user_id=3e8509e8358c462f89783dea99e00acc, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:38.742680 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d14e1-a143-43dc-8b0e-2e25bf868c6b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:38.781329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 819/1639] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:38 2022] PUT /identity/v3/projects/894f846ef7394eeebc695b698669cd56/users/3e8509e8358c462f89783dea99e00acc/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:38.789805 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:38.793120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5ab12-4562-407e-bc3c-35140e816b48 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:38.794068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5ab12-4562-407e-bc3c-35140e816b48 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:38.795244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5ab12-4562-407e-bc3c-35140e816b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:38.795410 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4100e-a1d3-4e27-855a-4eaa6ed3ce55 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:38.841441 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-a2c5ab12-4562-407e-bc3c-35140e816b48 None None] MFA Rules not processed for user `928d014648cd41d7b58e7e01845eb748`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:38.843437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a2c5ab12-4562-407e-bc3c-35140e816b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:38.885006 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5338b47-6785-4b26-a18a-5569a6932d3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:38.887409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5338b47-6785-4b26-a18a-5569a6932d3f service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:38.887914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5338b47-6785-4b26-a18a-5569a6932d3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:38.890072 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5338b47-6785-4b26-a18a-5569a6932d3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:38.990966 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 820/1640] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jul 21 05:43:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:38.992193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5ab12-4562-407e-bc3c-35140e816b48 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:39.021635 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5338b47-6785-4b26-a18a-5569a6932d3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:39.024199 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5338b47-6785-4b26-a18a-5569a6932d3f service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:39.047841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 821/1641] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:38 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:39.050988 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d6194-877d-4c48-9e0e-b49c2035c625 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:39.053374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d6194-877d-4c48-9e0e-b49c2035c625 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:39.053796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d6194-877d-4c48-9e0e-b49c2035c625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:39.108653 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5ab12-4562-407e-bc3c-35140e816b48 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:39.115245 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b68542-c11a-4aee-84f8-339f49216200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:39.116278 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b68542-c11a-4aee-84f8-339f49216200 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:39.116571 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b68542-c11a-4aee-84f8-339f49216200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:39.116896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b68542-c11a-4aee-84f8-339f49216200 admin admin] PATH_INFO: `/v3/projects/c19694f1c8f641359c4da57b70fffc90/users/0b60a90fc604451c848e908830c44afa/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:39.119583 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b68542-c11a-4aee-84f8-339f49216200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c19694f1c8f641359c4da57b70fffc90, user_id=0b60a90fc604451c848e908830c44afa, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:39.122566 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-1a0d6194-877d-4c48-9e0e-b49c2035c625 None None] MFA Rules not processed for user `3e8509e8358c462f89783dea99e00acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:39.123927 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b68542-c11a-4aee-84f8-339f49216200 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:39.127972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-1a0d6194-877d-4c48-9e0e-b49c2035c625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:39.142488 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 821/1642] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:38 2022] PUT /identity/v3/projects/c19694f1c8f641359c4da57b70fffc90/users/0b60a90fc604451c848e908830c44afa/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:39.144210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d4e05-95a0-4671-9afc-562bc93c36fa None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:39.144503 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d4e05-95a0-4671-9afc-562bc93c36fa None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:39.144786 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d4e05-95a0-4671-9afc-562bc93c36fa None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:39.147238 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 822/1643] 10.222.0.44 () {54 vars in 912 bytes} [Thu Jul 21 05:43:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:43:39.148074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d4e05-95a0-4671-9afc-562bc93c36fa None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:39.249061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 822/1644] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:39 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:39.249061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bc650-43b1-475b-ab70-cdf54e592dc3 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:39.249061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bc650-43b1-475b-ab70-cdf54e592dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:39.249061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bc650-43b1-475b-ab70-cdf54e592dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:39.279396 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-8b4bc650-43b1-475b-ab70-cdf54e592dc3 None None] MFA Rules not processed for user `0b60a90fc604451c848e908830c44afa`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:39.280941 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8b4bc650-43b1-475b-ab70-cdf54e592dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:39.315803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d4e05-95a0-4671-9afc-562bc93c36fa None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:39.317841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:39.319757 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f44ff-e46f-4be5-9f5a-98045cef3fda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:39.320426 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f44ff-e46f-4be5-9f5a-98045cef3fda service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:39.320708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f44ff-e46f-4be5-9f5a-98045cef3fda service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:39.320985 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f44ff-e46f-4be5-9f5a-98045cef3fda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:39.343898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:39.363551 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 823/1645] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:39 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:39.371897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3f3a6-6082-4049-9793-d534c2f86af5 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:39.372323 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3f3a6-6082-4049-9793-d534c2f86af5 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:39.372712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3f3a6-6082-4049-9793-d534c2f86af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:39.402500 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-2cd3f3a6-6082-4049-9793-d534c2f86af5 None None] MFA Rules not processed for user `928d014648cd41d7b58e7e01845eb748`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:39.403581 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-2cd3f3a6-6082-4049-9793-d534c2f86af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:39.462972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f44ff-e46f-4be5-9f5a-98045cef3fda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:39.464468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f44ff-e46f-4be5-9f5a-98045cef3fda service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:39.486160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 824/1646] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jul 21 05:43:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:39.487545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3f3a6-6082-4049-9793-d534c2f86af5 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:39.497996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 823/1647] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:39.498762 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3f3a6-6082-4049-9793-d534c2f86af5 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:39.502585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab03f83-e1fb-4f0a-a571-5fb226a3f05a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:39.503466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab03f83-e1fb-4f0a-a571-5fb226a3f05a service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:39.503868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab03f83-e1fb-4f0a-a571-5fb226a3f05a service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:39.504245 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab03f83-e1fb-4f0a-a571-5fb226a3f05a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:39.571113 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab03f83-e1fb-4f0a-a571-5fb226a3f05a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:39.572754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab03f83-e1fb-4f0a-a571-5fb226a3f05a service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:39.583949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 825/1648] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:39.904293 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f44ff-e46f-4be5-9f5a-98045cef3fda service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:39.913602 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f44ff-e46f-4be5-9f5a-98045cef3fda service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:39.922509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16abd58-da0f-4f7b-80b6-4b456560fa18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:39.923293 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16abd58-da0f-4f7b-80b6-4b456560fa18 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:39.923576 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16abd58-da0f-4f7b-80b6-4b456560fa18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:39.923899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16abd58-da0f-4f7b-80b6-4b456560fa18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:40.071398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16abd58-da0f-4f7b-80b6-4b456560fa18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:40.073570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16abd58-da0f-4f7b-80b6-4b456560fa18 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:40.094301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 824/1649] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:39 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:40.148250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab03f83-e1fb-4f0a-a571-5fb226a3f05a service cinder] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:40.165148 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab03f83-e1fb-4f0a-a571-5fb226a3f05a service cinder] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:40.169177 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca314f9-8596-45ec-a5e7-ea78b6386f1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:40.169821 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca314f9-8596-45ec-a5e7-ea78b6386f1f service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:40.170385 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca314f9-8596-45ec-a5e7-ea78b6386f1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:40.170385 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca314f9-8596-45ec-a5e7-ea78b6386f1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:40.193460 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca314f9-8596-45ec-a5e7-ea78b6386f1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:40.196712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca314f9-8596-45ec-a5e7-ea78b6386f1f service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:40.238720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 826/1650] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:40 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:40.292819 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16abd58-da0f-4f7b-80b6-4b456560fa18 service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:40.304546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16abd58-da0f-4f7b-80b6-4b456560fa18 service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:40.314105 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431164fb-7271-44d8-8dee-30935e8f2aba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:40.315242 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431164fb-7271-44d8-8dee-30935e8f2aba service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:40.317412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431164fb-7271-44d8-8dee-30935e8f2aba service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:40.318216 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431164fb-7271-44d8-8dee-30935e8f2aba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:40.348261 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431164fb-7271-44d8-8dee-30935e8f2aba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:40.353236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431164fb-7271-44d8-8dee-30935e8f2aba service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:40.396053 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 825/1651] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:40.640670 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca314f9-8596-45ec-a5e7-ea78b6386f1f service cinder] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:40.652660 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca314f9-8596-45ec-a5e7-ea78b6386f1f service cinder] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:40.671528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25c219-2d59-4747-b25c-23d9710d624d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:40.672255 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25c219-2d59-4747-b25c-23d9710d624d service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:40.672552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25c219-2d59-4747-b25c-23d9710d624d service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:40.672751 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25c219-2d59-4747-b25c-23d9710d624d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:40.805031 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:40.842995 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25c219-2d59-4747-b25c-23d9710d624d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:40.847305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25c219-2d59-4747-b25c-23d9710d624d service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:40.873021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 827/1652] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:40 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:43:41.224903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431164fb-7271-44d8-8dee-30935e8f2aba service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:41.235065 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431164fb-7271-44d8-8dee-30935e8f2aba service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:41.246375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86592dc-1bc2-4515-a719-dd937bea2df7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:41.249183 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86592dc-1bc2-4515-a719-dd937bea2df7 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:41.249899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86592dc-1bc2-4515-a719-dd937bea2df7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:41.250512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86592dc-1bc2-4515-a719-dd937bea2df7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:41.445269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86592dc-1bc2-4515-a719-dd937bea2df7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:41.449716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86592dc-1bc2-4515-a719-dd937bea2df7 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:41.498483 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 826/1653] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:41 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:42.212677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25c219-2d59-4747-b25c-23d9710d624d service cinder] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:42.354162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25c219-2d59-4747-b25c-23d9710d624d service cinder] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:42.365026 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a5895-80ca-4ed4-87a9-3f0415ae40a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:42.368253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a5895-80ca-4ed4-87a9-3f0415ae40a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:42.368759 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a5895-80ca-4ed4-87a9-3f0415ae40a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:42.369720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a5895-80ca-4ed4-87a9-3f0415ae40a1 admin admin] PATH_INFO: `/v3/users/45e9f4e991814209a322a78136e9bf19` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:42.373741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a5895-80ca-4ed4-87a9-3f0415ae40a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45e9f4e991814209a322a78136e9bf19)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:42.377789 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: WARNING py.warnings [None req-ae0a5895-80ca-4ed4-87a9-3f0415ae40a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 05:43:42.377789 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: warnings.warn(msg) Jul 21 05:43:42.377789 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: Jul 21 05:43:42.380799 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a5895-80ca-4ed4-87a9-3f0415ae40a1 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:42.518906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-ae0a5895-80ca-4ed4-87a9-3f0415ae40a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45e9f4e991814209a322a78136e9bf19'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:42.570309 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-ae0a5895-80ca-4ed4-87a9-3f0415ae40a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45e9f4e991814209a322a78136e9bf19'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:42.593342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-ae0a5895-80ca-4ed4-87a9-3f0415ae40a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45e9f4e991814209a322a78136e9bf19'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:42.596004 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-ae0a5895-80ca-4ed4-87a9-3f0415ae40a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45e9f4e991814209a322a78136e9bf19'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:42.597388 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-ae0a5895-80ca-4ed4-87a9-3f0415ae40a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45e9f4e991814209a322a78136e9bf19'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:42.651229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 828/1654] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:43:42 2022] DELETE /identity/v3/users/45e9f4e991814209a322a78136e9bf19 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:42.655806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb5ef7-b2d6-4f82-a483-acd325058306 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:42.656326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb5ef7-b2d6-4f82-a483-acd325058306 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:42.656408 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb5ef7-b2d6-4f82-a483-acd325058306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:42.702065 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-8cdb5ef7-b2d6-4f82-a483-acd325058306 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:42.704051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8cdb5ef7-b2d6-4f82-a483-acd325058306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:42.774507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:42.921020 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 827/1655] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:43.091838 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a5895-80ca-4ed4-87a9-3f0415ae40a1 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:43.093768 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ae0a5895-80ca-4ed4-87a9-3f0415ae40a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:43.267766 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a5895-80ca-4ed4-87a9-3f0415ae40a1 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:43.283768 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439ceea-60b5-43d1-8f51-debcf574104c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:43.284508 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439ceea-60b5-43d1-8f51-debcf574104c service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:43.284788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439ceea-60b5-43d1-8f51-debcf574104c service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:43.285767 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3439ceea-60b5-43d1-8f51-debcf574104c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:43.306831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439ceea-60b5-43d1-8f51-debcf574104c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:43.311799 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3439ceea-60b5-43d1-8f51-debcf574104c service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:43.359085 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 829/1656] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:44.928591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900f4bb-dd03-4f60-9a8e-d73bc9a81b9b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:44.929208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900f4bb-dd03-4f60-9a8e-d73bc9a81b9b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:44.929208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900f4bb-dd03-4f60-9a8e-d73bc9a81b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:44.966439 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-2900f4bb-dd03-4f60-9a8e-d73bc9a81b9b None None] MFA Rules not processed for user `0cbf797171144c139c72c516cea59bb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:44.967817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2900f4bb-dd03-4f60-9a8e-d73bc9a81b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:45.108600 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 828/1657] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:45.125523 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439ceea-60b5-43d1-8f51-debcf574104c service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:45.128008 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3439ceea-60b5-43d1-8f51-debcf574104c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:45.134005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:45.193804 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900f4bb-dd03-4f60-9a8e-d73bc9a81b9b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:45.195531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2900f4bb-dd03-4f60-9a8e-d73bc9a81b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:45.211868 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900f4bb-dd03-4f60-9a8e-d73bc9a81b9b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:45.220136 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468316f5-5c3d-41c1-b85e-979632c547fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:45.221663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468316f5-5c3d-41c1-b85e-979632c547fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:45.222154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468316f5-5c3d-41c1-b85e-979632c547fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:45.222862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468316f5-5c3d-41c1-b85e-979632c547fc admin admin] PATH_INFO: `/v3/projects/28238eceb52e42d985df53b2ba8a42df` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:45.224623 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468316f5-5c3d-41c1-b85e-979632c547fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=28238eceb52e42d985df53b2ba8a42df)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:45.226019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: WARNING py.warnings [None req-468316f5-5c3d-41c1-b85e-979632c547fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 21 05:43:45.226019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: warnings.warn(msg) Jul 21 05:43:45.226019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: Jul 21 05:43:45.230027 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468316f5-5c3d-41c1-b85e-979632c547fc admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:45.243459 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3439ceea-60b5-43d1-8f51-debcf574104c service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:45.259272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9edc67d-03c1-4554-84bb-8db453f6da8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:45.260381 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9edc67d-03c1-4554-84bb-8db453f6da8a service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:45.260940 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9edc67d-03c1-4554-84bb-8db453f6da8a service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:45.261490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9edc67d-03c1-4554-84bb-8db453f6da8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:45.277164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9edc67d-03c1-4554-84bb-8db453f6da8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:45.281867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9edc67d-03c1-4554-84bb-8db453f6da8a service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:45.307942 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 830/1658] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:45.313356 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-468316f5-5c3d-41c1-b85e-979632c547fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28238eceb52e42d985df53b2ba8a42df', 'request_id': 'req-468316f5-5c3d-41c1-b85e-979632c547fc'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:45.324182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-468316f5-5c3d-41c1-b85e-979632c547fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28238eceb52e42d985df53b2ba8a42df', 'request_id': 'req-468316f5-5c3d-41c1-b85e-979632c547fc'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:45.341022 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-468316f5-5c3d-41c1-b85e-979632c547fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28238eceb52e42d985df53b2ba8a42df', 'request_id': 'req-468316f5-5c3d-41c1-b85e-979632c547fc'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:45.386195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-468316f5-5c3d-41c1-b85e-979632c547fc admin admin] The token cache is being invalidate because project 28238eceb52e42d985df53b2ba8a42df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:43:45.386528 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-468316f5-5c3d-41c1-b85e-979632c547fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:45.388865 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-468316f5-5c3d-41c1-b85e-979632c547fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:45.397210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7111e-7d79-4edd-86b0-c4eda9fa61a6 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:45.397499 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7111e-7d79-4edd-86b0-c4eda9fa61a6 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:45.398137 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7111e-7d79-4edd-86b0-c4eda9fa61a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:45.401028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 829/1659] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:43:45 2022] DELETE /identity/v3/projects/28238eceb52e42d985df53b2ba8a42df => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:45.453036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-19e7111e-7d79-4edd-86b0-c4eda9fa61a6 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:45.454492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-19e7111e-7d79-4edd-86b0-c4eda9fa61a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:45.595785 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 831/1660] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:45.597339 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28390f6-944d-4b0d-860b-7b9e60635f9a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:45.597645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28390f6-944d-4b0d-860b-7b9e60635f9a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:45.597926 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28390f6-944d-4b0d-860b-7b9e60635f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:45.626964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-c28390f6-944d-4b0d-860b-7b9e60635f9a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:45.628619 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c28390f6-944d-4b0d-860b-7b9e60635f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:45.670220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 830/1661] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:45.672985 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7111e-7d79-4edd-86b0-c4eda9fa61a6 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:45.704643 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7111e-7d79-4edd-86b0-c4eda9fa61a6 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:45.730032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b201b-8ac4-4f1b-8878-8b27d1b1caee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:45.731978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b201b-8ac4-4f1b-8878-8b27d1b1caee admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:45.733523 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b201b-8ac4-4f1b-8878-8b27d1b1caee admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:45.741606 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b201b-8ac4-4f1b-8878-8b27d1b1caee admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:45.741606 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b201b-8ac4-4f1b-8878-8b27d1b1caee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:43:45.741606 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b201b-8ac4-4f1b-8878-8b27d1b1caee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:45.745438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b201b-8ac4-4f1b-8878-8b27d1b1caee admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:45.777223 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 832/1662] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:43:45.780709 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28390f6-944d-4b0d-860b-7b9e60635f9a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:45.795661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28390f6-944d-4b0d-860b-7b9e60635f9a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:45.810662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176cc81b-aa3b-4d0b-9b63-c426b16ff2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:45.811486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cc81b-aa3b-4d0b-9b63-c426b16ff2dd admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:45.811781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cc81b-aa3b-4d0b-9b63-c426b16ff2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:45.812157 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cc81b-aa3b-4d0b-9b63-c426b16ff2dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:45.812881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176cc81b-aa3b-4d0b-9b63-c426b16ff2dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:45.815486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176cc81b-aa3b-4d0b-9b63-c426b16ff2dd admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:45.864914 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 831/1663] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:43:45 2022] POST /identity/v3/projects => generated 369 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:45.869059 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b201b-8ac4-4f1b-8878-8b27d1b1caee admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:45.884692 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b201b-8ac4-4f1b-8878-8b27d1b1caee admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:46.023642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640f851-e12a-4070-bca6-e41000d22ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:46.024358 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640f851-e12a-4070-bca6-e41000d22ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:46.024641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640f851-e12a-4070-bca6-e41000d22ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:46.024914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640f851-e12a-4070-bca6-e41000d22ae7 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:46.025623 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640f851-e12a-4070-bca6-e41000d22ae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:46.027668 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640f851-e12a-4070-bca6-e41000d22ae7 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:46.060595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 833/1664] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:43:45 2022] POST /identity/v3/users => generated 285 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:46.065134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176cc81b-aa3b-4d0b-9b63-c426b16ff2dd admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:46.080466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:46.090887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176cc81b-aa3b-4d0b-9b63-c426b16ff2dd admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:46.103845 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c7047-7ee3-4269-8598-eddc14997e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:46.104867 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c7047-7ee3-4269-8598-eddc14997e35 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:46.105347 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c7047-7ee3-4269-8598-eddc14997e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:46.105789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c7047-7ee3-4269-8598-eddc14997e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:46.106745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c7047-7ee3-4269-8598-eddc14997e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:46.108658 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c7047-7ee3-4269-8598-eddc14997e35 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:46.137544 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 832/1665] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:46.139127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640f851-e12a-4070-bca6-e41000d22ae7 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:46.148768 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:46.152835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640f851-e12a-4070-bca6-e41000d22ae7 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:46.167453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e3e5c-33d8-4c87-a3ea-8ae4644e44aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:46.168386 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e3e5c-33d8-4c87-a3ea-8ae4644e44aa admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:46.168872 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e3e5c-33d8-4c87-a3ea-8ae4644e44aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:46.169387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e3e5c-33d8-4c87-a3ea-8ae4644e44aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:46.170321 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e3e5c-33d8-4c87-a3ea-8ae4644e44aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:46.172702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e3e5c-33d8-4c87-a3ea-8ae4644e44aa admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:46.186248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 834/1666] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:46.190850 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c7047-7ee3-4269-8598-eddc14997e35 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:46.205325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c7047-7ee3-4269-8598-eddc14997e35 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:46.220126 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7640fbe-e18b-4163-a35b-5c3b96044d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:46.224556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7640fbe-e18b-4163-a35b-5c3b96044d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:46.225092 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7640fbe-e18b-4163-a35b-5c3b96044d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:46.225538 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7640fbe-e18b-4163-a35b-5c3b96044d60 admin admin] PATH_INFO: `/v3/projects/8a4fdfb457c14965b05efd67dceb6e7a/users/f1f7104d975e4180a2955c555397b9fe/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:46.262612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7640fbe-e18b-4163-a35b-5c3b96044d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a4fdfb457c14965b05efd67dceb6e7a, user_id=f1f7104d975e4180a2955c555397b9fe, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:46.265632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7640fbe-e18b-4163-a35b-5c3b96044d60 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:46.318388 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 833/1667] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:46 2022] PUT /identity/v3/projects/8a4fdfb457c14965b05efd67dceb6e7a/users/f1f7104d975e4180a2955c555397b9fe/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:43:46.320308 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70142d8-9889-4665-81db-5fe3536627bd None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:46.320603 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70142d8-9889-4665-81db-5fe3536627bd None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:46.320890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70142d8-9889-4665-81db-5fe3536627bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:46.371274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-f70142d8-9889-4665-81db-5fe3536627bd None None] MFA Rules not processed for user `f1f7104d975e4180a2955c555397b9fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:46.373077 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f70142d8-9889-4665-81db-5fe3536627bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:46.471895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 835/1668] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:46 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:46.476870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7640fbe-e18b-4163-a35b-5c3b96044d60 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:46.635804 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7640fbe-e18b-4163-a35b-5c3b96044d60 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:46.644780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2319c05-00fe-470c-9a6f-b20f4a320a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:46.650751 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2319c05-00fe-470c-9a6f-b20f4a320a37 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:46.651372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2319c05-00fe-470c-9a6f-b20f4a320a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:46.651761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2319c05-00fe-470c-9a6f-b20f4a320a37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:46.652939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2319c05-00fe-470c-9a6f-b20f4a320a37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:46.656273 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2319c05-00fe-470c-9a6f-b20f4a320a37 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:46.723235 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 834/1669] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:43:46 2022] POST /identity/v3/projects => generated 369 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:46.727879 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70142d8-9889-4665-81db-5fe3536627bd None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:46.758010 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70142d8-9889-4665-81db-5fe3536627bd None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:46.902973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c3107-7068-4227-9df8-bd760566c7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:46.907304 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c3107-7068-4227-9df8-bd760566c7e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:46.907704 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c3107-7068-4227-9df8-bd760566c7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:46.908088 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c3107-7068-4227-9df8-bd760566c7e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:46.909238 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c3107-7068-4227-9df8-bd760566c7e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:46.913332 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c3107-7068-4227-9df8-bd760566c7e6 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:46.965209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 836/1670] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:43:46 2022] POST /identity/v3/users => generated 291 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:46.968734 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2319c05-00fe-470c-9a6f-b20f4a320a37 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:46.983439 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2319c05-00fe-470c-9a6f-b20f4a320a37 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:46.995619 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cc2fc-0652-4d0d-90d4-f301b1ebc3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:46.996255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cc2fc-0652-4d0d-90d4-f301b1ebc3de admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:46.996544 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cc2fc-0652-4d0d-90d4-f301b1ebc3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:46.996826 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cc2fc-0652-4d0d-90d4-f301b1ebc3de admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:46.997580 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cc2fc-0652-4d0d-90d4-f301b1ebc3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:47.007175 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cc2fc-0652-4d0d-90d4-f301b1ebc3de admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:47.037536 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 835/1671] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:47.038745 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c3107-7068-4227-9df8-bd760566c7e6 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:47.052367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c3107-7068-4227-9df8-bd760566c7e6 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:47.063766 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9c9f9-f2bf-4902-88c2-6ab7b7ae9a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:47.066552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9c9f9-f2bf-4902-88c2-6ab7b7ae9a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:47.066857 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9c9f9-f2bf-4902-88c2-6ab7b7ae9a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:47.067780 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9c9f9-f2bf-4902-88c2-6ab7b7ae9a19 admin admin] PATH_INFO: `/v3/projects/58b5602c19314fdca66ab5bd577881b1/users/7e66e72603fc4351a90c0a09871346a4/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:47.115166 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9c9f9-f2bf-4902-88c2-6ab7b7ae9a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58b5602c19314fdca66ab5bd577881b1, user_id=7e66e72603fc4351a90c0a09871346a4, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:47.120466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9c9f9-f2bf-4902-88c2-6ab7b7ae9a19 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:47.158353 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 837/1672] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:47 2022] PUT /identity/v3/projects/58b5602c19314fdca66ab5bd577881b1/users/7e66e72603fc4351a90c0a09871346a4/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:43:47.162755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cc2fc-0652-4d0d-90d4-f301b1ebc3de admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:47.178941 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cc2fc-0652-4d0d-90d4-f301b1ebc3de admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:47.364748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d5139-4ee6-43ea-964f-550b5917bcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:47.365524 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d5139-4ee6-43ea-964f-550b5917bcb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:47.365812 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d5139-4ee6-43ea-964f-550b5917bcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:47.366094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d5139-4ee6-43ea-964f-550b5917bcb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:47.368681 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d5139-4ee6-43ea-964f-550b5917bcb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:47.377003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d5139-4ee6-43ea-964f-550b5917bcb5 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:47.408688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 836/1673] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:47.412250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9c9f9-f2bf-4902-88c2-6ab7b7ae9a19 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:47.427325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9c9f9-f2bf-4902-88c2-6ab7b7ae9a19 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:47.438720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091fd0c-7621-40d1-984d-4b9da84632e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:47.442347 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091fd0c-7621-40d1-984d-4b9da84632e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:47.442644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091fd0c-7621-40d1-984d-4b9da84632e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:47.442940 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091fd0c-7621-40d1-984d-4b9da84632e7 admin admin] PATH_INFO: `/v3/projects/58b5602c19314fdca66ab5bd577881b1/users/7e66e72603fc4351a90c0a09871346a4/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:47.450984 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091fd0c-7621-40d1-984d-4b9da84632e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58b5602c19314fdca66ab5bd577881b1, user_id=7e66e72603fc4351a90c0a09871346a4, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:47.462748 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091fd0c-7621-40d1-984d-4b9da84632e7 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:47.502695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 838/1674] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:47 2022] PUT /identity/v3/projects/58b5602c19314fdca66ab5bd577881b1/users/7e66e72603fc4351a90c0a09871346a4/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:43:47.507867 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3563a-8670-4a8e-91dd-f89ad682e9d0 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:47.508171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3563a-8670-4a8e-91dd-f89ad682e9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:47.508463 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3563a-8670-4a8e-91dd-f89ad682e9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:47.571466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-28b3563a-8670-4a8e-91dd-f89ad682e9d0 None None] MFA Rules not processed for user `7e66e72603fc4351a90c0a09871346a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:47.574835 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-28b3563a-8670-4a8e-91dd-f89ad682e9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:47.655907 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:47.783840 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 837/1675] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:47 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:47.796223 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091fd0c-7621-40d1-984d-4b9da84632e7 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:47.799245 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d091fd0c-7621-40d1-984d-4b9da84632e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:47.866426 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d51cbd-3b42-4ce4-ba90-ceefce0a9b65 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:47.866725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d51cbd-3b42-4ce4-ba90-ceefce0a9b65 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:47.867005 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d51cbd-3b42-4ce4-ba90-ceefce0a9b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:47.906665 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:47.940424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-99d51cbd-3b42-4ce4-ba90-ceefce0a9b65 None None] MFA Rules not processed for user `d01f138c8f0d4940af6737d2576a7326`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:47.941124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091fd0c-7621-40d1-984d-4b9da84632e7 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:47.941985 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-99d51cbd-3b42-4ce4-ba90-ceefce0a9b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:47.949791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445fc34-38bb-47d2-bb4d-e73a4bec96e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:47.953510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445fc34-38bb-47d2-bb4d-e73a4bec96e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:47.953919 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445fc34-38bb-47d2-bb4d-e73a4bec96e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:47.954250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445fc34-38bb-47d2-bb4d-e73a4bec96e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:47.999734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445fc34-38bb-47d2-bb4d-e73a4bec96e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:48.001297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445fc34-38bb-47d2-bb4d-e73a4bec96e2 service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:48.017617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 839/1676] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:47 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:43:48.103506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 838/1677] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:48.116825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bbafc-202e-4c1d-b197-391bf26f0c1e None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:48.117125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bbafc-202e-4c1d-b197-391bf26f0c1e None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:48.117412 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bbafc-202e-4c1d-b197-391bf26f0c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:48.127680 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d51cbd-3b42-4ce4-ba90-ceefce0a9b65 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:48.131629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-99d51cbd-3b42-4ce4-ba90-ceefce0a9b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:48.157665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d51cbd-3b42-4ce4-ba90-ceefce0a9b65 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:48.162564 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-491bbafc-202e-4c1d-b197-391bf26f0c1e None None] MFA Rules not processed for user `d01f138c8f0d4940af6737d2576a7326`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:48.165265 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-491bbafc-202e-4c1d-b197-391bf26f0c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:48.167734 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1823e7-09ca-4ca9-a301-4502e690379b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:48.169990 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1823e7-09ca-4ca9-a301-4502e690379b service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:48.170450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1823e7-09ca-4ca9-a301-4502e690379b service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:48.171236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1823e7-09ca-4ca9-a301-4502e690379b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:48.187862 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 840/1678] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:48.201679 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bbafc-202e-4c1d-b197-391bf26f0c1e None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:48.203522 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-491bbafc-202e-4c1d-b197-391bf26f0c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:48.346445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1823e7-09ca-4ca9-a301-4502e690379b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:48.348027 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1823e7-09ca-4ca9-a301-4502e690379b service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:48.384551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 839/1679] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:48 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:48.406178 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bbafc-202e-4c1d-b197-391bf26f0c1e None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:48.437515 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf20cc6-fcf3-4ed2-8c3c-4f8bc3cbfea5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:48.438203 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf20cc6-fcf3-4ed2-8c3c-4f8bc3cbfea5 service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:48.438492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf20cc6-fcf3-4ed2-8c3c-4f8bc3cbfea5 service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:48.449324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf20cc6-fcf3-4ed2-8c3c-4f8bc3cbfea5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:48.493118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf20cc6-fcf3-4ed2-8c3c-4f8bc3cbfea5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:48.494906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf20cc6-fcf3-4ed2-8c3c-4f8bc3cbfea5 service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:48.536643 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 841/1680] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:43:49.671557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09143288-ee51-43b1-8bea-88a0af708bf2 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:49.671996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09143288-ee51-43b1-8bea-88a0af708bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:49.672128 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09143288-ee51-43b1-8bea-88a0af708bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:49.702855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-09143288-ee51-43b1-8bea-88a0af708bf2 None None] MFA Rules not processed for user `0cbf797171144c139c72c516cea59bb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:49.703314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a16e3c-b2eb-47a0-b968-5d30b66d3177 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:49.703774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a16e3c-b2eb-47a0-b968-5d30b66d3177 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:49.703880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a16e3c-b2eb-47a0-b968-5d30b66d3177 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:49.705440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-09143288-ee51-43b1-8bea-88a0af708bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:49.710525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 842/1681] 10.222.0.44 () {54 vars in 912 bytes} [Thu Jul 21 05:43:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 21 05:43:49.716262 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3adbc-8101-4e64-91ae-e53f2ab96efa None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:49.717321 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3adbc-8101-4e64-91ae-e53f2ab96efa None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:49.717808 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3adbc-8101-4e64-91ae-e53f2ab96efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:49.781519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:49.799826 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 840/1682] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:49.840688 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-9eb3adbc-8101-4e64-91ae-e53f2ab96efa None None] MFA Rules not processed for user `e07508a9fa9146528620b7f8abec8549`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:49.841844 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9eb3adbc-8101-4e64-91ae-e53f2ab96efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:49.971550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 843/1683] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jul 21 05:43:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:43:49.977755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09143288-ee51-43b1-8bea-88a0af708bf2 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:49.979601 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-09143288-ee51-43b1-8bea-88a0af708bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:49.998637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09143288-ee51-43b1-8bea-88a0af708bf2 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:50.006167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a5538d-b418-4946-b540-92d9518a0518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:50.016315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5538d-b418-4946-b540-92d9518a0518 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:50.016967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5538d-b418-4946-b540-92d9518a0518 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:50.017629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5538d-b418-4946-b540-92d9518a0518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:50.027889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5538d-b418-4946-b540-92d9518a0518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:50.031148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a5538d-b418-4946-b540-92d9518a0518 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:50.045184 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 841/1684] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:50.249721 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3adbc-8101-4e64-91ae-e53f2ab96efa None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:50.251230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9eb3adbc-8101-4e64-91ae-e53f2ab96efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:50.369202 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:50.400774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3adbc-8101-4e64-91ae-e53f2ab96efa None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:50.414979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dfc4e5-15b0-414a-acb3-089f4fe3aaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:50.417007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfc4e5-15b0-414a-acb3-089f4fe3aaf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:50.418022 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfc4e5-15b0-414a-acb3-089f4fe3aaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:50.419212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dfc4e5-15b0-414a-acb3-089f4fe3aaf1 admin admin] PATH_INFO: `/v3/users/762c9249de0f45d88242e9f4e27ab246` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:50.424337 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dfc4e5-15b0-414a-acb3-089f4fe3aaf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=762c9249de0f45d88242e9f4e27ab246)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:50.433982 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dfc4e5-15b0-414a-acb3-089f4fe3aaf1 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:50.542180 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-10dfc4e5-15b0-414a-acb3-089f4fe3aaf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '762c9249de0f45d88242e9f4e27ab246'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:50.570064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-10dfc4e5-15b0-414a-acb3-089f4fe3aaf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '762c9249de0f45d88242e9f4e27ab246'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:50.602278 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-10dfc4e5-15b0-414a-acb3-089f4fe3aaf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '762c9249de0f45d88242e9f4e27ab246'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:50.603158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-10dfc4e5-15b0-414a-acb3-089f4fe3aaf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '762c9249de0f45d88242e9f4e27ab246'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:50.603847 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-10dfc4e5-15b0-414a-acb3-089f4fe3aaf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '762c9249de0f45d88242e9f4e27ab246'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:50.630725 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 844/1685] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:43:50 2022] DELETE /identity/v3/users/762c9249de0f45d88242e9f4e27ab246 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:50.635223 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d0e12-4469-4563-9633-4eea17f75a2a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:50.635734 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d0e12-4469-4563-9633-4eea17f75a2a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:50.636164 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2d0e12-4469-4563-9633-4eea17f75a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:50.695256 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-9b2d0e12-4469-4563-9633-4eea17f75a2a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:50.700527 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-9b2d0e12-4469-4563-9633-4eea17f75a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:50.859305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c0ac2-dd24-45fc-aef6-51dcc9a0d234 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:50.859824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c0ac2-dd24-45fc-aef6-51dcc9a0d234 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:50.860306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c0ac2-dd24-45fc-aef6-51dcc9a0d234 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:50.862168 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 845/1686] 10.222.0.44 () {54 vars in 912 bytes} [Thu Jul 21 05:43:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:43:50.874203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 842/1687] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:50.883385 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76053ce3-65f1-41d6-a80e-ac83142b617f None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:50.883736 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76053ce3-65f1-41d6-a80e-ac83142b617f None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:50.884034 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76053ce3-65f1-41d6-a80e-ac83142b617f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:50.912362 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d0e12-4469-4563-9633-4eea17f75a2a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:50.917434 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-9b2d0e12-4469-4563-9633-4eea17f75a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:50.933641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-76053ce3-65f1-41d6-a80e-ac83142b617f None None] MFA Rules not processed for user `e07508a9fa9146528620b7f8abec8549`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:50.935147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-76053ce3-65f1-41d6-a80e-ac83142b617f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:51.064445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2d0e12-4469-4563-9633-4eea17f75a2a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:51.079556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9ea1e-8cd9-400a-bc80-dfba7816f240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:51.080466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9ea1e-8cd9-400a-bc80-dfba7816f240 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:51.083232 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9ea1e-8cd9-400a-bc80-dfba7816f240 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:51.083716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9ea1e-8cd9-400a-bc80-dfba7816f240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:51.093453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 846/1688] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jul 21 05:43:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:51.097520 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf041f4f-6820-4e04-bf1f-c2925dc23601 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:51.097944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf041f4f-6820-4e04-bf1f-c2925dc23601 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:51.099778 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf041f4f-6820-4e04-bf1f-c2925dc23601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:51.117514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a9ea1e-8cd9-400a-bc80-dfba7816f240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:51.121165 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a9ea1e-8cd9-400a-bc80-dfba7816f240 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:51.142812 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 843/1689] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:51.146903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9ea1e-8cd9-400a-bc80-dfba7816f240 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:51.187477 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9ea1e-8cd9-400a-bc80-dfba7816f240 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:51.192679 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae188492-e6d2-4acd-b6b0-c657957d20a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:51.193433 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae188492-e6d2-4acd-b6b0-c657957d20a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:51.195332 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae188492-e6d2-4acd-b6b0-c657957d20a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:51.195693 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae188492-e6d2-4acd-b6b0-c657957d20a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:51.200575 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-bf041f4f-6820-4e04-bf1f-c2925dc23601 None None] MFA Rules not processed for user `563f1367093e493bae87a21374a3c2f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:51.203651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-bf041f4f-6820-4e04-bf1f-c2925dc23601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:51.209279 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae188492-e6d2-4acd-b6b0-c657957d20a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:51.210877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae188492-e6d2-4acd-b6b0-c657957d20a9 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:51.234220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 844/1690] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:51.326261 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 847/1691] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:51.337350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6f517-6527-4989-9172-81eedf075fe2 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:51.337886 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6f517-6527-4989-9172-81eedf075fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:51.338231 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6f517-6527-4989-9172-81eedf075fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:51.400432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-7cd6f517-6527-4989-9172-81eedf075fe2 None None] MFA Rules not processed for user `563f1367093e493bae87a21374a3c2f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:51.404129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-7cd6f517-6527-4989-9172-81eedf075fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:51.438830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 845/1692] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:51.444316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8b475-229e-431d-bdf1-f86a5de56536 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:51.444615 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8b475-229e-431d-bdf1-f86a5de56536 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:51.444899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8b475-229e-431d-bdf1-f86a5de56536 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:51.446824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 848/1693] 10.222.0.44 () {54 vars in 961 bytes} [Thu Jul 21 05:43:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:43:51.453369 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7a6c2-d356-4ac1-9e80-7506a1865b8c None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:51.453789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7a6c2-d356-4ac1-9e80-7506a1865b8c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:51.454218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7a6c2-d356-4ac1-9e80-7506a1865b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:51.497465 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-e7c7a6c2-d356-4ac1-9e80-7506a1865b8c None None] MFA Rules not processed for user `03d868598e39448881b43d299253d821`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:51.502160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e7c7a6c2-d356-4ac1-9e80-7506a1865b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:51.663412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 846/1694] 10.222.0.44 () {58 vars in 1058 bytes} [Thu Jul 21 05:43:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:51.673227 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8b475-229e-431d-bdf1-f86a5de56536 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:51.701828 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8b475-229e-431d-bdf1-f86a5de56536 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:51.714986 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97422b87-9664-4c54-b740-b64568c2b98d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:51.716189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97422b87-9664-4c54-b740-b64568c2b98d service placement] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:51.716697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97422b87-9664-4c54-b740-b64568c2b98d service placement] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:51.717175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97422b87-9664-4c54-b740-b64568c2b98d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:51.722867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-97422b87-9664-4c54-b740-b64568c2b98d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:51.752103 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97422b87-9664-4c54-b740-b64568c2b98d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:51.753963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97422b87-9664-4c54-b740-b64568c2b98d service placement] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:51.762585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:51.775231 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 849/1695] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:52.055485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7a6c2-d356-4ac1-9e80-7506a1865b8c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:52.058659 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e7c7a6c2-d356-4ac1-9e80-7506a1865b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:52.068617 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:52.271425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97422b87-9664-4c54-b740-b64568c2b98d service placement] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:52.272988 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-97422b87-9664-4c54-b740-b64568c2b98d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:52.280760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7a6c2-d356-4ac1-9e80-7506a1865b8c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:52.305214 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97422b87-9664-4c54-b740-b64568c2b98d service placement] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:52.314808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba044d-66f9-47d7-8a55-7448c24f5eaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:52.319334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba044d-66f9-47d7-8a55-7448c24f5eaf service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:52.319654 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba044d-66f9-47d7-8a55-7448c24f5eaf service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:52.319939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba044d-66f9-47d7-8a55-7448c24f5eaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:52.323271 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692d7ed-23a8-403c-9063-e79c73fbe58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:52.324168 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692d7ed-23a8-403c-9063-e79c73fbe58c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:52.324611 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692d7ed-23a8-403c-9063-e79c73fbe58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:52.325025 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692d7ed-23a8-403c-9063-e79c73fbe58c admin admin] PATH_INFO: `/v3/projects/80c9f6c2ae394669be9a3a7efbb400fe` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:52.329768 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692d7ed-23a8-403c-9063-e79c73fbe58c admin admin] RBAC: Authorizing `identity:delete_project(project_id=80c9f6c2ae394669be9a3a7efbb400fe)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:52.332617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692d7ed-23a8-403c-9063-e79c73fbe58c admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:52.346758 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba044d-66f9-47d7-8a55-7448c24f5eaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:52.357083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba044d-66f9-47d7-8a55-7448c24f5eaf service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:52.424902 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 847/1696] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:52.456660 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b692d7ed-23a8-403c-9063-e79c73fbe58c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80c9f6c2ae394669be9a3a7efbb400fe', 'request_id': 'req-b692d7ed-23a8-403c-9063-e79c73fbe58c'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:52.469205 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b692d7ed-23a8-403c-9063-e79c73fbe58c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80c9f6c2ae394669be9a3a7efbb400fe', 'request_id': 'req-b692d7ed-23a8-403c-9063-e79c73fbe58c'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:52.487792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b692d7ed-23a8-403c-9063-e79c73fbe58c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80c9f6c2ae394669be9a3a7efbb400fe', 'request_id': 'req-b692d7ed-23a8-403c-9063-e79c73fbe58c'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:52.512968 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b692d7ed-23a8-403c-9063-e79c73fbe58c admin admin] The token cache is being invalidate because project 80c9f6c2ae394669be9a3a7efbb400fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:43:52.513486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b692d7ed-23a8-403c-9063-e79c73fbe58c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:52.514434 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b692d7ed-23a8-403c-9063-e79c73fbe58c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:52.531119 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 850/1697] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:43:52 2022] DELETE /identity/v3/projects/80c9f6c2ae394669be9a3a7efbb400fe => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:52.533094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba044d-66f9-47d7-8a55-7448c24f5eaf service nova] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:52.614228 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba044d-66f9-47d7-8a55-7448c24f5eaf service nova] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:52.810177 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f63ce0-c95a-4865-a522-1c35a5b78e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:52.812479 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f63ce0-c95a-4865-a522-1c35a5b78e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:52.812841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f63ce0-c95a-4865-a522-1c35a5b78e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:52.813231 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f63ce0-c95a-4865-a522-1c35a5b78e40 admin admin] PATH_INFO: `/v3/users/0b60a90fc604451c848e908830c44afa` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:52.816943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f63ce0-c95a-4865-a522-1c35a5b78e40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b60a90fc604451c848e908830c44afa)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:52.820019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f63ce0-c95a-4865-a522-1c35a5b78e40 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:52.982613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-47f63ce0-c95a-4865-a522-1c35a5b78e40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b60a90fc604451c848e908830c44afa'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:53.010658 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-47f63ce0-c95a-4865-a522-1c35a5b78e40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b60a90fc604451c848e908830c44afa'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:53.037870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-47f63ce0-c95a-4865-a522-1c35a5b78e40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b60a90fc604451c848e908830c44afa'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:53.039863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-47f63ce0-c95a-4865-a522-1c35a5b78e40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b60a90fc604451c848e908830c44afa'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:53.041104 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-47f63ce0-c95a-4865-a522-1c35a5b78e40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b60a90fc604451c848e908830c44afa'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:53.095386 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 848/1698] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:43:52 2022] DELETE /identity/v3/users/0b60a90fc604451c848e908830c44afa => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:53.932153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692d7ed-23a8-403c-9063-e79c73fbe58c admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:53.933799 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b692d7ed-23a8-403c-9063-e79c73fbe58c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:53.972196 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692d7ed-23a8-403c-9063-e79c73fbe58c admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:53.972994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f63ce0-c95a-4865-a522-1c35a5b78e40 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:53.977426 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-47f63ce0-c95a-4865-a522-1c35a5b78e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:54.030426 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:54.171783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bde12d-f85c-4400-bc8e-b5770e307798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:54.174056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bde12d-f85c-4400-bc8e-b5770e307798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:54.174689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bde12d-f85c-4400-bc8e-b5770e307798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:54.175242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bde12d-f85c-4400-bc8e-b5770e307798 admin admin] PATH_INFO: `/v3/projects/c19694f1c8f641359c4da57b70fffc90` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:54.177952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bde12d-f85c-4400-bc8e-b5770e307798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c19694f1c8f641359c4da57b70fffc90)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:54.180289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bde12d-f85c-4400-bc8e-b5770e307798 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:54.217657 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f63ce0-c95a-4865-a522-1c35a5b78e40 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:54.225748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4706505-d56e-49e6-a20f-534da0d79bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:54.227168 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4706505-d56e-49e6-a20f-534da0d79bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:54.228076 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4706505-d56e-49e6-a20f-534da0d79bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:54.229087 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4706505-d56e-49e6-a20f-534da0d79bd8 admin admin] PATH_INFO: `/v3/users/f1f7104d975e4180a2955c555397b9fe` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:54.230714 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4706505-d56e-49e6-a20f-534da0d79bd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1f7104d975e4180a2955c555397b9fe)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:54.252393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4706505-d56e-49e6-a20f-534da0d79bd8 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:54.312104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b8bde12d-f85c-4400-bc8e-b5770e307798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c19694f1c8f641359c4da57b70fffc90', 'request_id': 'req-b8bde12d-f85c-4400-bc8e-b5770e307798'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:54.335497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b8bde12d-f85c-4400-bc8e-b5770e307798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c19694f1c8f641359c4da57b70fffc90', 'request_id': 'req-b8bde12d-f85c-4400-bc8e-b5770e307798'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:54.358117 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b8bde12d-f85c-4400-bc8e-b5770e307798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c19694f1c8f641359c4da57b70fffc90', 'request_id': 'req-b8bde12d-f85c-4400-bc8e-b5770e307798'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:54.381020 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e4706505-d56e-49e6-a20f-534da0d79bd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1f7104d975e4180a2955c555397b9fe'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:54.391209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b8bde12d-f85c-4400-bc8e-b5770e307798 admin admin] The token cache is being invalidate because project c19694f1c8f641359c4da57b70fffc90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:43:54.391581 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b8bde12d-f85c-4400-bc8e-b5770e307798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:54.393885 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b8bde12d-f85c-4400-bc8e-b5770e307798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:54.404813 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 851/1699] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:43:53 2022] DELETE /identity/v3/projects/c19694f1c8f641359c4da57b70fffc90 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:54.406360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576f283-893b-4659-94ca-8882c958cc90 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:54.406705 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576f283-893b-4659-94ca-8882c958cc90 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:54.407006 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576f283-893b-4659-94ca-8882c958cc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:54.425034 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e4706505-d56e-49e6-a20f-534da0d79bd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1f7104d975e4180a2955c555397b9fe'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:54.452418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e4706505-d56e-49e6-a20f-534da0d79bd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1f7104d975e4180a2955c555397b9fe'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:54.456262 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e4706505-d56e-49e6-a20f-534da0d79bd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1f7104d975e4180a2955c555397b9fe'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:54.457773 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e4706505-d56e-49e6-a20f-534da0d79bd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1f7104d975e4180a2955c555397b9fe'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:54.476362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-0576f283-893b-4659-94ca-8882c958cc90 None None] MFA Rules not processed for user `563f1367093e493bae87a21374a3c2f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:54.477931 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0576f283-893b-4659-94ca-8882c958cc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:54.489702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 849/1700] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:43:53 2022] DELETE /identity/v3/users/f1f7104d975e4180a2955c555397b9fe => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:54.491536 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6655ede-5613-444d-abec-04c064c07fec None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:54.491955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6655ede-5613-444d-abec-04c064c07fec None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:54.492909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6655ede-5613-444d-abec-04c064c07fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:54.498364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:54.520813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-c6655ede-5613-444d-abec-04c064c07fec None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:54.522589 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c6655ede-5613-444d-abec-04c064c07fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:54.729928 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 852/1701] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:54.743046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4759ae-d019-498c-bb62-58f5ba6f2a13 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:54.745780 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4759ae-d019-498c-bb62-58f5ba6f2a13 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:54.746091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4759ae-d019-498c-bb62-58f5ba6f2a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:54.794808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 850/1702] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:54.798856 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da6ba6-9f30-4cf8-9435-62206e62d210 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:54.799357 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da6ba6-9f30-4cf8-9435-62206e62d210 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:54.799864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da6ba6-9f30-4cf8-9435-62206e62d210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:54.849288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-5d4759ae-d019-498c-bb62-58f5ba6f2a13 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:54.855876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5d4759ae-d019-498c-bb62-58f5ba6f2a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:54.874563 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 853/1703] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:54.898142 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4759ae-d019-498c-bb62-58f5ba6f2a13 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:54.900824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5d4759ae-d019-498c-bb62-58f5ba6f2a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:54.911281 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-70da6ba6-9f30-4cf8-9435-62206e62d210 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:54.920187 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-70da6ba6-9f30-4cf8-9435-62206e62d210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:54.986262 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 851/1704] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:54.992095 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da6ba6-9f30-4cf8-9435-62206e62d210 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:55.034040 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da6ba6-9f30-4cf8-9435-62206e62d210 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:55.055999 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692351ba-efab-4894-81c2-61dac035fe2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:55.057899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692351ba-efab-4894-81c2-61dac035fe2c admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:55.058374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692351ba-efab-4894-81c2-61dac035fe2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:55.058770 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692351ba-efab-4894-81c2-61dac035fe2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:55.059512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692351ba-efab-4894-81c2-61dac035fe2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:43:55.059825 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692351ba-efab-4894-81c2-61dac035fe2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:55.061489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692351ba-efab-4894-81c2-61dac035fe2c admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:55.113547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 852/1705] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:43:55.118550 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692351ba-efab-4894-81c2-61dac035fe2c admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:55.146456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692351ba-efab-4894-81c2-61dac035fe2c admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:55.147256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4759ae-d019-498c-bb62-58f5ba6f2a13 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:55.163838 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adbeef-6525-443b-b025-5e399b1d6939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:55.164808 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adbeef-6525-443b-b025-5e399b1d6939 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:55.164808 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adbeef-6525-443b-b025-5e399b1d6939 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:55.165089 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adbeef-6525-443b-b025-5e399b1d6939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:55.175602 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff1917-1109-4430-8a2e-5bd2f4193b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:55.176412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff1917-1109-4430-8a2e-5bd2f4193b05 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:55.176412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff1917-1109-4430-8a2e-5bd2f4193b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:55.176777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff1917-1109-4430-8a2e-5bd2f4193b05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:55.177316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff1917-1109-4430-8a2e-5bd2f4193b05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:55.189813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ff1917-1109-4430-8a2e-5bd2f4193b05 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:55.192808 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36adbeef-6525-443b-b025-5e399b1d6939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:55.195944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36adbeef-6525-443b-b025-5e399b1d6939 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:55.211335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:55.212782 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 854/1706] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:55.280796 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 853/1707] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:43:55 2022] POST /identity/v3/projects => generated 369 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:43:55.284807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adbeef-6525-443b-b025-5e399b1d6939 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:55.298564 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adbeef-6525-443b-b025-5e399b1d6939 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:55.385486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3bba0-2b63-4b7b-ac4f-83371044d008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:55.386521 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3bba0-2b63-4b7b-ac4f-83371044d008 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:55.386858 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3bba0-2b63-4b7b-ac4f-83371044d008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:55.387254 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3bba0-2b63-4b7b-ac4f-83371044d008 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:55.388198 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3bba0-2b63-4b7b-ac4f-83371044d008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:55.391383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3bba0-2b63-4b7b-ac4f-83371044d008 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:55.439508 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 855/1708] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:43:55 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:55.444935 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff1917-1109-4430-8a2e-5bd2f4193b05 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:55.459146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff1917-1109-4430-8a2e-5bd2f4193b05 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:55.464942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c3517-0efc-453a-bb8d-03ed2616d30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:55.465783 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c3517-0efc-453a-bb8d-03ed2616d30c admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:55.466163 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c3517-0efc-453a-bb8d-03ed2616d30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:55.466767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c3517-0efc-453a-bb8d-03ed2616d30c admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:55.467412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7c3517-0efc-453a-bb8d-03ed2616d30c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:55.469191 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7c3517-0efc-453a-bb8d-03ed2616d30c admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:55.496655 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 854/1709] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:55.502759 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3bba0-2b63-4b7b-ac4f-83371044d008 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:55.515951 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3bba0-2b63-4b7b-ac4f-83371044d008 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:55.536104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2031f-c024-4d4e-be7c-9fffb7e2d541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:55.537018 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2031f-c024-4d4e-be7c-9fffb7e2d541 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:55.537311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2031f-c024-4d4e-be7c-9fffb7e2d541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:55.537628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2031f-c024-4d4e-be7c-9fffb7e2d541 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:55.540074 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2031f-c024-4d4e-be7c-9fffb7e2d541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:55.543792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2031f-c024-4d4e-be7c-9fffb7e2d541 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:55.558603 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 856/1710] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:55.566545 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c3517-0efc-453a-bb8d-03ed2616d30c admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:55.590000 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c3517-0efc-453a-bb8d-03ed2616d30c admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:55.604549 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d7064-a86e-4b0d-8ef7-6c4b83c8e57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:55.605463 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d7064-a86e-4b0d-8ef7-6c4b83c8e57c admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:55.605806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d7064-a86e-4b0d-8ef7-6c4b83c8e57c admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:55.606082 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d7064-a86e-4b0d-8ef7-6c4b83c8e57c admin admin] PATH_INFO: `/v3/projects/9ef278c5009947c98fffdeac5c734a78/users/ea26809d37e84bf2af75075fe6ebd4de/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:55.665046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d7064-a86e-4b0d-8ef7-6c4b83c8e57c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ef278c5009947c98fffdeac5c734a78, user_id=ea26809d37e84bf2af75075fe6ebd4de, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:55.680029 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d7064-a86e-4b0d-8ef7-6c4b83c8e57c admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:55.724079 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 855/1711] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:55 2022] PUT /identity/v3/projects/9ef278c5009947c98fffdeac5c734a78/users/ea26809d37e84bf2af75075fe6ebd4de/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:55.735472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee5045-36a7-407f-8158-940c134437e6 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:55.735862 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee5045-36a7-407f-8158-940c134437e6 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:55.736186 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee5045-36a7-407f-8158-940c134437e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:55.803206 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-d7ee5045-36a7-407f-8158-940c134437e6 None None] MFA Rules not processed for user `ea26809d37e84bf2af75075fe6ebd4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:55.804629 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d7ee5045-36a7-407f-8158-940c134437e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:55.842229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:55.877398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d7064-a86e-4b0d-8ef7-6c4b83c8e57c admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:55.879414 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e12d7064-a86e-4b0d-8ef7-6c4b83c8e57c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:55.974323 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 857/1712] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:55.979061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee5045-36a7-407f-8158-940c134437e6 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:56.085783 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d7064-a86e-4b0d-8ef7-6c4b83c8e57c admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:56.121097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee5045-36a7-407f-8158-940c134437e6 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:56.131007 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6be442-b860-4cd6-ae79-76ddf43fbb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:56.131921 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6be442-b860-4cd6-ae79-76ddf43fbb63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:56.132424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6be442-b860-4cd6-ae79-76ddf43fbb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:56.132917 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6be442-b860-4cd6-ae79-76ddf43fbb63 admin admin] PATH_INFO: `/v3/projects/8a4fdfb457c14965b05efd67dceb6e7a` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:56.134389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6be442-b860-4cd6-ae79-76ddf43fbb63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a4fdfb457c14965b05efd67dceb6e7a)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:56.137078 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6be442-b860-4cd6-ae79-76ddf43fbb63 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:56.142792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddb713-181c-4e79-9724-157b6c7f7d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:56.145941 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddb713-181c-4e79-9724-157b6c7f7d84 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:56.146257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddb713-181c-4e79-9724-157b6c7f7d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:56.147236 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddb713-181c-4e79-9724-157b6c7f7d84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:56.151783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fddb713-181c-4e79-9724-157b6c7f7d84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:56.163080 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fddb713-181c-4e79-9724-157b6c7f7d84 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:56.233368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 858/1713] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:43:55 2022] POST /identity/v3/projects => generated 367 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:56.235984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-fd6be442-b860-4cd6-ae79-76ddf43fbb63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a4fdfb457c14965b05efd67dceb6e7a', 'request_id': 'req-fd6be442-b860-4cd6-ae79-76ddf43fbb63'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:56.237617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddb713-181c-4e79-9724-157b6c7f7d84 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:56.248533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-fd6be442-b860-4cd6-ae79-76ddf43fbb63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a4fdfb457c14965b05efd67dceb6e7a', 'request_id': 'req-fd6be442-b860-4cd6-ae79-76ddf43fbb63'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:56.268081 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-fd6be442-b860-4cd6-ae79-76ddf43fbb63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a4fdfb457c14965b05efd67dceb6e7a', 'request_id': 'req-fd6be442-b860-4cd6-ae79-76ddf43fbb63'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:56.297291 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-fd6be442-b860-4cd6-ae79-76ddf43fbb63 admin admin] The token cache is being invalidate because project 8a4fdfb457c14965b05efd67dceb6e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:43:56.297944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-fd6be442-b860-4cd6-ae79-76ddf43fbb63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:56.299369 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-fd6be442-b860-4cd6-ae79-76ddf43fbb63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:56.309367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 856/1714] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:43:55 2022] DELETE /identity/v3/projects/8a4fdfb457c14965b05efd67dceb6e7a => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:43:56.312905 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6be442-b860-4cd6-ae79-76ddf43fbb63 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:56.317506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-fd6be442-b860-4cd6-ae79-76ddf43fbb63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:56.430024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddb713-181c-4e79-9724-157b6c7f7d84 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:56.435804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1eec4-c939-4993-9c9d-8dd5a1d29f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:56.436680 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1eec4-c939-4993-9c9d-8dd5a1d29f39 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:56.437041 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1eec4-c939-4993-9c9d-8dd5a1d29f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:56.437413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1eec4-c939-4993-9c9d-8dd5a1d29f39 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:56.438524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b1eec4-c939-4993-9c9d-8dd5a1d29f39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:56.441258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b1eec4-c939-4993-9c9d-8dd5a1d29f39 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:56.442070 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:56.496995 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 859/1715] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:43:56 2022] POST /identity/v3/users => generated 290 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:56.503258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1eec4-c939-4993-9c9d-8dd5a1d29f39 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:56.505102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6be442-b860-4cd6-ae79-76ddf43fbb63 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:56.507704 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-80b1eec4-c939-4993-9c9d-8dd5a1d29f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:56.517203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3397f-0cde-4a0f-b4b3-b614a8da598d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:56.526368 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3397f-0cde-4a0f-b4b3-b614a8da598d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:56.526897 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3397f-0cde-4a0f-b4b3-b614a8da598d admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:56.527365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3397f-0cde-4a0f-b4b3-b614a8da598d admin admin] PATH_INFO: `/v3/users/7e66e72603fc4351a90c0a09871346a4` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:56.532961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3397f-0cde-4a0f-b4b3-b614a8da598d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e66e72603fc4351a90c0a09871346a4)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:56.538270 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b3397f-0cde-4a0f-b4b3-b614a8da598d admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:56.543473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1eec4-c939-4993-9c9d-8dd5a1d29f39 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:56.556948 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43da04-2a70-41a2-b1d0-93500aca75be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:56.558275 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43da04-2a70-41a2-b1d0-93500aca75be admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:56.559046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43da04-2a70-41a2-b1d0-93500aca75be admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:56.559532 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43da04-2a70-41a2-b1d0-93500aca75be admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:56.562285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43da04-2a70-41a2-b1d0-93500aca75be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:56.566544 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea43da04-2a70-41a2-b1d0-93500aca75be admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:56.582397 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 860/1716] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:56.588992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43da04-2a70-41a2-b1d0-93500aca75be admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:56.608849 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea43da04-2a70-41a2-b1d0-93500aca75be admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:56.615397 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77568597-e564-4153-b595-3bdbcefa50f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:56.616306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77568597-e564-4153-b595-3bdbcefa50f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:56.616602 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77568597-e564-4153-b595-3bdbcefa50f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:56.616882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77568597-e564-4153-b595-3bdbcefa50f1 admin admin] PATH_INFO: `/v3/projects/ca7f43c19b8443f78d3cce5d5cbba66d/users/8cf9ca7b97f04558a2a9116041d9dfc4/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:56.649639 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77568597-e564-4153-b595-3bdbcefa50f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca7f43c19b8443f78d3cce5d5cbba66d, user_id=8cf9ca7b97f04558a2a9116041d9dfc4, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:56.652612 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77568597-e564-4153-b595-3bdbcefa50f1 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:56.668022 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-21b3397f-0cde-4a0f-b4b3-b614a8da598d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e66e72603fc4351a90c0a09871346a4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:56.675769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:56.681878 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 861/1717] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:56 2022] PUT /identity/v3/projects/ca7f43c19b8443f78d3cce5d5cbba66d/users/8cf9ca7b97f04558a2a9116041d9dfc4/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:43:56.685439 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77568597-e564-4153-b595-3bdbcefa50f1 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:56.688385 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-21b3397f-0cde-4a0f-b4b3-b614a8da598d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e66e72603fc4351a90c0a09871346a4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:56.697503 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-21b3397f-0cde-4a0f-b4b3-b614a8da598d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e66e72603fc4351a90c0a09871346a4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:56.698212 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-21b3397f-0cde-4a0f-b4b3-b614a8da598d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e66e72603fc4351a90c0a09871346a4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:56.698967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-21b3397f-0cde-4a0f-b4b3-b614a8da598d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e66e72603fc4351a90c0a09871346a4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:56.716832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 857/1718] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:43:56 2022] DELETE /identity/v3/users/7e66e72603fc4351a90c0a09871346a4 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:56.758454 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77568597-e564-4153-b595-3bdbcefa50f1 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:56.826113 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d62e5c-c136-4d2b-b6f2-727513fb4be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:56.826810 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d62e5c-c136-4d2b-b6f2-727513fb4be3 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:56.827587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d62e5c-c136-4d2b-b6f2-727513fb4be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:56.827587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d62e5c-c136-4d2b-b6f2-727513fb4be3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:56.828318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d62e5c-c136-4d2b-b6f2-727513fb4be3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:56.829923 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d62e5c-c136-4d2b-b6f2-727513fb4be3 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:56.845924 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 862/1719] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:43:56.847455 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3397f-0cde-4a0f-b4b3-b614a8da598d admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:56.849587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-21b3397f-0cde-4a0f-b4b3-b614a8da598d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:56.866905 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3397f-0cde-4a0f-b4b3-b614a8da598d admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:56.872724 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b841f52-5e45-4cf3-a731-78f0e5646f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:56.873360 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b841f52-5e45-4cf3-a731-78f0e5646f3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:56.873814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b841f52-5e45-4cf3-a731-78f0e5646f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:56.874196 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b841f52-5e45-4cf3-a731-78f0e5646f3d admin admin] PATH_INFO: `/v3/projects/ca7f43c19b8443f78d3cce5d5cbba66d/users/8cf9ca7b97f04558a2a9116041d9dfc4/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:56.876461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b841f52-5e45-4cf3-a731-78f0e5646f3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca7f43c19b8443f78d3cce5d5cbba66d, user_id=8cf9ca7b97f04558a2a9116041d9dfc4, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:56.886189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b841f52-5e45-4cf3-a731-78f0e5646f3d admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:56.898333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 858/1720] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:56 2022] PUT /identity/v3/projects/ca7f43c19b8443f78d3cce5d5cbba66d/users/8cf9ca7b97f04558a2a9116041d9dfc4/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:56.909050 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d3aa6-701d-4fdd-88d3-d392e373d597 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:56.909424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d3aa6-701d-4fdd-88d3-d392e373d597 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:56.909771 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d3aa6-701d-4fdd-88d3-d392e373d597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:56.938170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-dc8d3aa6-701d-4fdd-88d3-d392e373d597 None None] MFA Rules not processed for user `8cf9ca7b97f04558a2a9116041d9dfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:56.939218 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-dc8d3aa6-701d-4fdd-88d3-d392e373d597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:57.009634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 863/1721] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:56 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:57.021745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b841f52-5e45-4cf3-a731-78f0e5646f3d admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:57.023802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-3b841f52-5e45-4cf3-a731-78f0e5646f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:57.040449 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d3aa6-701d-4fdd-88d3-d392e373d597 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:57.042085 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-dc8d3aa6-701d-4fdd-88d3-d392e373d597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:57.075964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:57.088358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b841f52-5e45-4cf3-a731-78f0e5646f3d admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:57.091890 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba7e9d-75c9-492f-b57a-8103c6ce2515 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:57.092643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba7e9d-75c9-492f-b57a-8103c6ce2515 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:57.092924 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba7e9d-75c9-492f-b57a-8103c6ce2515 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:57.093246 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba7e9d-75c9-492f-b57a-8103c6ce2515 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:57.102267 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba7e9d-75c9-492f-b57a-8103c6ce2515 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:57.103745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba7e9d-75c9-492f-b57a-8103c6ce2515 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:57.110504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d3aa6-701d-4fdd-88d3-d392e373d597 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:57.113461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 859/1722] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:57 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:57.117026 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef324d70-b758-4170-a7e7-6aedfddd4aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:57.117899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef324d70-b758-4170-a7e7-6aedfddd4aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:57.118177 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef324d70-b758-4170-a7e7-6aedfddd4aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:57.118442 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef324d70-b758-4170-a7e7-6aedfddd4aae admin admin] PATH_INFO: `/v3/projects/58b5602c19314fdca66ab5bd577881b1` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:57.119629 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef324d70-b758-4170-a7e7-6aedfddd4aae admin admin] RBAC: Authorizing `identity:delete_project(project_id=58b5602c19314fdca66ab5bd577881b1)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:57.121545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef324d70-b758-4170-a7e7-6aedfddd4aae admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:57.176721 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-ef324d70-b758-4170-a7e7-6aedfddd4aae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58b5602c19314fdca66ab5bd577881b1', 'request_id': 'req-ef324d70-b758-4170-a7e7-6aedfddd4aae'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:57.185294 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-ef324d70-b758-4170-a7e7-6aedfddd4aae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58b5602c19314fdca66ab5bd577881b1', 'request_id': 'req-ef324d70-b758-4170-a7e7-6aedfddd4aae'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:57.191891 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-ef324d70-b758-4170-a7e7-6aedfddd4aae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58b5602c19314fdca66ab5bd577881b1', 'request_id': 'req-ef324d70-b758-4170-a7e7-6aedfddd4aae'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:57.218203 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-ef324d70-b758-4170-a7e7-6aedfddd4aae admin admin] The token cache is being invalidate because project 58b5602c19314fdca66ab5bd577881b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:43:57.218731 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-ef324d70-b758-4170-a7e7-6aedfddd4aae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:57.220386 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-ef324d70-b758-4170-a7e7-6aedfddd4aae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:57.223149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 864/1723] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:43:57 2022] DELETE /identity/v3/projects/58b5602c19314fdca66ab5bd577881b1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:57.227930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fce40-0356-43d9-947e-3ba437ae477b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:57.228222 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fce40-0356-43d9-947e-3ba437ae477b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:57.228498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fce40-0356-43d9-947e-3ba437ae477b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:57.248335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-606fce40-0356-43d9-947e-3ba437ae477b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:57.249729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-606fce40-0356-43d9-947e-3ba437ae477b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:57.308057 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:57.319613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 860/1724] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:57.327858 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e088f-6469-46d4-8afb-4eece713a174 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:57.328149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e088f-6469-46d4-8afb-4eece713a174 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:57.328427 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e088f-6469-46d4-8afb-4eece713a174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:57.347654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-a96e088f-6469-46d4-8afb-4eece713a174 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:57.348928 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a96e088f-6469-46d4-8afb-4eece713a174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:57.357931 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 865/1725] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:43:57.362336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fce40-0356-43d9-947e-3ba437ae477b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:57.377069 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fce40-0356-43d9-947e-3ba437ae477b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:57.382256 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cc442-c3b8-4a10-a7b3-9a55ae828f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:57.383073 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cc442-c3b8-4a10-a7b3-9a55ae828f24 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:57.383469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cc442-c3b8-4a10-a7b3-9a55ae828f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:57.383884 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cc442-c3b8-4a10-a7b3-9a55ae828f24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:57.384729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cc442-c3b8-4a10-a7b3-9a55ae828f24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:43:57.385182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cc442-c3b8-4a10-a7b3-9a55ae828f24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:57.387412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cc442-c3b8-4a10-a7b3-9a55ae828f24 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:57.401145 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 861/1726] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:43:57.405760 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e088f-6469-46d4-8afb-4eece713a174 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:57.418883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e088f-6469-46d4-8afb-4eece713a174 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:57.424646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbbf3a-2b20-4e54-84fb-8029409686f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:57.425338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbbf3a-2b20-4e54-84fb-8029409686f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:57.425760 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbbf3a-2b20-4e54-84fb-8029409686f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:57.425939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbbf3a-2b20-4e54-84fb-8029409686f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:57.426653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbbbf3a-2b20-4e54-84fb-8029409686f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:57.430368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbbbf3a-2b20-4e54-84fb-8029409686f4 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:57.480316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 866/1727] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:43:57 2022] POST /identity/v3/projects => generated 409 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:57.484878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cc442-c3b8-4a10-a7b3-9a55ae828f24 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:57.507192 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cc442-c3b8-4a10-a7b3-9a55ae828f24 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:57.587086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92031d-51e4-4556-becf-4ffa1276429e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:57.588261 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92031d-51e4-4556-becf-4ffa1276429e admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:57.588865 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92031d-51e4-4556-becf-4ffa1276429e admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:57.589104 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92031d-51e4-4556-becf-4ffa1276429e admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:57.589802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92031d-51e4-4556-becf-4ffa1276429e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:57.593866 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92031d-51e4-4556-becf-4ffa1276429e admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:57.621431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 862/1728] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:43:57 2022] POST /identity/v3/users => generated 305 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:57.625737 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbbf3a-2b20-4e54-84fb-8029409686f4 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:57.639846 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbbf3a-2b20-4e54-84fb-8029409686f4 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:57.655188 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86023c4a-b719-4569-8316-3a206e8b3656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:57.656015 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86023c4a-b719-4569-8316-3a206e8b3656 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:57.656213 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86023c4a-b719-4569-8316-3a206e8b3656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:57.656548 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86023c4a-b719-4569-8316-3a206e8b3656 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:57.657428 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86023c4a-b719-4569-8316-3a206e8b3656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:57.659339 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86023c4a-b719-4569-8316-3a206e8b3656 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:57.686363 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 867/1729] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:57.687705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92031d-51e4-4556-becf-4ffa1276429e admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:57.703068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92031d-51e4-4556-becf-4ffa1276429e admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:57.709542 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae783e-5249-4c08-bbc0-a9b1a4bab415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:57.710332 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae783e-5249-4c08-bbc0-a9b1a4bab415 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:57.710600 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae783e-5249-4c08-bbc0-a9b1a4bab415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:57.711101 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae783e-5249-4c08-bbc0-a9b1a4bab415 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:57.712103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae783e-5249-4c08-bbc0-a9b1a4bab415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:57.713865 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae783e-5249-4c08-bbc0-a9b1a4bab415 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:57.736236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 863/1730] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:43:57.737405 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86023c4a-b719-4569-8316-3a206e8b3656 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:57.749882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86023c4a-b719-4569-8316-3a206e8b3656 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:57.758453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8286668-6b59-4c3f-8ece-b11e41f81334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:57.759410 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8286668-6b59-4c3f-8ece-b11e41f81334 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:57.760016 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8286668-6b59-4c3f-8ece-b11e41f81334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:57.769694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8286668-6b59-4c3f-8ece-b11e41f81334 admin admin] PATH_INFO: `/v3/projects/733bf3e2212e47c1a7051b532f812e61/users/a031645424364c9c9658724093f0caf5/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:57.791488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8286668-6b59-4c3f-8ece-b11e41f81334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=733bf3e2212e47c1a7051b532f812e61, user_id=a031645424364c9c9658724093f0caf5, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:57.794499 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8286668-6b59-4c3f-8ece-b11e41f81334 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:57.819068 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 868/1731] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:57 2022] PUT /identity/v3/projects/733bf3e2212e47c1a7051b532f812e61/users/a031645424364c9c9658724093f0caf5/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:57.827568 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6105d54-9632-45e0-a12f-bbbca4cffa43 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:57.827879 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6105d54-9632-45e0-a12f-bbbca4cffa43 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:57.828134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6105d54-9632-45e0-a12f-bbbca4cffa43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:57.858018 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-a6105d54-9632-45e0-a12f-bbbca4cffa43 None None] MFA Rules not processed for user `a031645424364c9c9658724093f0caf5`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:57.859456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a6105d54-9632-45e0-a12f-bbbca4cffa43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:57.944930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 864/1732] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:57 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:57.949802 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8286668-6b59-4c3f-8ece-b11e41f81334 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:58.015342 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:58.021130 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8286668-6b59-4c3f-8ece-b11e41f81334 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:58.034636 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c8ef1-b1ea-42fb-92cb-210e5ab0fcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:58.036183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c8ef1-b1ea-42fb-92cb-210e5ab0fcad admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:58.036843 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c8ef1-b1ea-42fb-92cb-210e5ab0fcad admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:58.037379 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c8ef1-b1ea-42fb-92cb-210e5ab0fcad admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:58.038862 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c8ef1-b1ea-42fb-92cb-210e5ab0fcad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:58.041085 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c8ef1-b1ea-42fb-92cb-210e5ab0fcad admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:58.072024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 869/1733] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:43:57 2022] POST /identity/v3/projects => generated 407 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:43:58.076030 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6105d54-9632-45e0-a12f-bbbca4cffa43 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:58.088063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6105d54-9632-45e0-a12f-bbbca4cffa43 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:58.131986 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:58.146286 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e90ff-0387-4f84-9344-0a01e59d1b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:58.146976 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e90ff-0387-4f84-9344-0a01e59d1b8b admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:58.147303 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e90ff-0387-4f84-9344-0a01e59d1b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:58.147581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e90ff-0387-4f84-9344-0a01e59d1b8b admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:58.148282 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e90ff-0387-4f84-9344-0a01e59d1b8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:58.154739 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e90ff-0387-4f84-9344-0a01e59d1b8b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:58.171795 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 865/1734] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:43:58 2022] POST /identity/v3/users => generated 310 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:43:58.175918 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c8ef1-b1ea-42fb-92cb-210e5ab0fcad admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:58.187282 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c8ef1-b1ea-42fb-92cb-210e5ab0fcad admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:58.192968 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf142a2-72e8-4365-87b8-f528bea214b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:58.193650 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf142a2-72e8-4365-87b8-f528bea214b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:58.193938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf142a2-72e8-4365-87b8-f528bea214b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:58.194214 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf142a2-72e8-4365-87b8-f528bea214b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:58.195169 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf142a2-72e8-4365-87b8-f528bea214b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:58.196790 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf142a2-72e8-4365-87b8-f528bea214b1 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:58.211470 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 870/1735] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:58.212674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e90ff-0387-4f84-9344-0a01e59d1b8b admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:58.224402 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e90ff-0387-4f84-9344-0a01e59d1b8b admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:58.229045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56ae16-6a38-4486-9884-b876cc55e340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:58.229951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56ae16-6a38-4486-9884-b876cc55e340 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:58.230241 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56ae16-6a38-4486-9884-b876cc55e340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:58.230523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56ae16-6a38-4486-9884-b876cc55e340 admin admin] PATH_INFO: `/v3/projects/64a03e578f904d00a1e7d4c48293e2ed/users/a1362159194a4b948d3ff67e085d2b8e/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:58.246100 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56ae16-6a38-4486-9884-b876cc55e340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a03e578f904d00a1e7d4c48293e2ed, user_id=a1362159194a4b948d3ff67e085d2b8e, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:58.249601 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56ae16-6a38-4486-9884-b876cc55e340 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:58.263815 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 866/1736] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:58 2022] PUT /identity/v3/projects/64a03e578f904d00a1e7d4c48293e2ed/users/a1362159194a4b948d3ff67e085d2b8e/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:58.268707 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf142a2-72e8-4365-87b8-f528bea214b1 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:58.279726 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf142a2-72e8-4365-87b8-f528bea214b1 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:58.353184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122a9b1-997b-41ca-a1de-08fec082e821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:58.353862 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122a9b1-997b-41ca-a1de-08fec082e821 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:58.354146 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122a9b1-997b-41ca-a1de-08fec082e821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:58.354424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122a9b1-997b-41ca-a1de-08fec082e821 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:58.355184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122a9b1-997b-41ca-a1de-08fec082e821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:58.356796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122a9b1-997b-41ca-a1de-08fec082e821 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:58.368177 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 871/1737] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:43:58.375246 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56ae16-6a38-4486-9884-b876cc55e340 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:58.394405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56ae16-6a38-4486-9884-b876cc55e340 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:58.396856 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a4921-25e2-4060-bc9d-0c23b7507e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:58.397763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a4921-25e2-4060-bc9d-0c23b7507e89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:58.398052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a4921-25e2-4060-bc9d-0c23b7507e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:58.398336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a4921-25e2-4060-bc9d-0c23b7507e89 admin admin] PATH_INFO: `/v3/projects/64a03e578f904d00a1e7d4c48293e2ed/users/a1362159194a4b948d3ff67e085d2b8e/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:58.400479 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a4921-25e2-4060-bc9d-0c23b7507e89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a03e578f904d00a1e7d4c48293e2ed, user_id=a1362159194a4b948d3ff67e085d2b8e, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:58.403439 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a4921-25e2-4060-bc9d-0c23b7507e89 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:58.423050 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 867/1738] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:43:58 2022] PUT /identity/v3/projects/64a03e578f904d00a1e7d4c48293e2ed/users/a1362159194a4b948d3ff67e085d2b8e/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:58.431143 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e6fd9-804e-4320-be2f-7ba7e78be7aa None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:58.431520 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e6fd9-804e-4320-be2f-7ba7e78be7aa None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:58.431843 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e6fd9-804e-4320-be2f-7ba7e78be7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:58.461348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-6b8e6fd9-804e-4320-be2f-7ba7e78be7aa None None] MFA Rules not processed for user `a1362159194a4b948d3ff67e085d2b8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:58.462574 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-6b8e6fd9-804e-4320-be2f-7ba7e78be7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:58.546142 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 872/1739] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:58 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:58.560197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a4921-25e2-4060-bc9d-0c23b7507e89 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:58.562578 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b48a4921-25e2-4060-bc9d-0c23b7507e89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:58.624086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a4921-25e2-4060-bc9d-0c23b7507e89 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:58.627714 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41a3da-af38-479d-b780-b84c7068927e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:58.628372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41a3da-af38-479d-b780-b84c7068927e service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:58.628657 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41a3da-af38-479d-b780-b84c7068927e service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:58.628959 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41a3da-af38-479d-b780-b84c7068927e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:58.638403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41a3da-af38-479d-b780-b84c7068927e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:58.639907 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41a3da-af38-479d-b780-b84c7068927e service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:58.656670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 868/1740] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:58 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:58.738646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e6fd9-804e-4320-be2f-7ba7e78be7aa None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:58.740115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-6b8e6fd9-804e-4320-be2f-7ba7e78be7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:58.812440 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:58.844181 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e6fd9-804e-4320-be2f-7ba7e78be7aa None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:58.853937 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ccae3-dbe9-478b-aaa6-62d5b0459087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:58.854890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ccae3-dbe9-478b-aaa6-62d5b0459087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:58.855178 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ccae3-dbe9-478b-aaa6-62d5b0459087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:58.858271 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ccae3-dbe9-478b-aaa6-62d5b0459087 admin admin] PATH_INFO: `/v3/users/ea26809d37e84bf2af75075fe6ebd4de` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:58.860567 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ccae3-dbe9-478b-aaa6-62d5b0459087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea26809d37e84bf2af75075fe6ebd4de)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:58.866732 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ccae3-dbe9-478b-aaa6-62d5b0459087 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:58.954468 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-aa5ccae3-dbe9-478b-aaa6-62d5b0459087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea26809d37e84bf2af75075fe6ebd4de'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:58.968773 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41a3da-af38-479d-b780-b84c7068927e service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:58.973264 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-aa5ccae3-dbe9-478b-aaa6-62d5b0459087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea26809d37e84bf2af75075fe6ebd4de'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:58.992491 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41a3da-af38-479d-b780-b84c7068927e service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:58.992626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-aa5ccae3-dbe9-478b-aaa6-62d5b0459087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea26809d37e84bf2af75075fe6ebd4de'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:58.992626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-aa5ccae3-dbe9-478b-aaa6-62d5b0459087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea26809d37e84bf2af75075fe6ebd4de'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:58.992626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-aa5ccae3-dbe9-478b-aaa6-62d5b0459087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea26809d37e84bf2af75075fe6ebd4de'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:43:58.998233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5b856-3fbb-4fed-abce-1d427998e45c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:58.998951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5b856-3fbb-4fed-abce-1d427998e45c service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:58.999330 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5b856-3fbb-4fed-abce-1d427998e45c service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:58.999661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5b856-3fbb-4fed-abce-1d427998e45c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:59.004694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8cf5b856-3fbb-4fed-abce-1d427998e45c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:59.008441 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 873/1741] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:43:58 2022] DELETE /identity/v3/users/ea26809d37e84bf2af75075fe6ebd4de => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:43:59.016710 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffaa6a1-5bef-44a7-8f5a-b9f4bc8affbc None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:59.016710 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffaa6a1-5bef-44a7-8f5a-b9f4bc8affbc None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:59.017157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffaa6a1-5bef-44a7-8f5a-b9f4bc8affbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:59.038190 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-3ffaa6a1-5bef-44a7-8f5a-b9f4bc8affbc None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:43:59.040245 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3ffaa6a1-5bef-44a7-8f5a-b9f4bc8affbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:59.122473 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5b856-3fbb-4fed-abce-1d427998e45c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:59.124530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf5b856-3fbb-4fed-abce-1d427998e45c service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:59.138643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 869/1742] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:58 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:43:59.147173 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 874/1743] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:43:59.169000 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5b856-3fbb-4fed-abce-1d427998e45c service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:43:59.171433 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8cf5b856-3fbb-4fed-abce-1d427998e45c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:43:59.248061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:43:59.280307 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5b856-3fbb-4fed-abce-1d427998e45c service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:43:59.357917 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb212cb-8dd8-40c7-ae71-cc9865c5af7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:43:59.358603 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb212cb-8dd8-40c7-ae71-cc9865c5af7a service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:43:59.358891 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb212cb-8dd8-40c7-ae71-cc9865c5af7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:43:59.359179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb212cb-8dd8-40c7-ae71-cc9865c5af7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:43:59.373499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb212cb-8dd8-40c7-ae71-cc9865c5af7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:43:59.375575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb212cb-8dd8-40c7-ae71-cc9865c5af7a service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:43:59.388628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 870/1744] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:43:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:00.084645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb212cb-8dd8-40c7-ae71-cc9865c5af7a service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:00.085325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d846bb5-7240-4001-8072-79c35d2adbd0 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:00.089025 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d846bb5-7240-4001-8072-79c35d2adbd0 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:00.089538 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d846bb5-7240-4001-8072-79c35d2adbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:00.091225 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1fb212cb-8dd8-40c7-ae71-cc9865c5af7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:00.126057 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb212cb-8dd8-40c7-ae71-cc9865c5af7a service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:00.139467 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45780b21-9a9c-454b-8fdc-b342bcc67905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:00.144351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45780b21-9a9c-454b-8fdc-b342bcc67905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:00.144951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45780b21-9a9c-454b-8fdc-b342bcc67905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:00.145402 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45780b21-9a9c-454b-8fdc-b342bcc67905 admin admin] PATH_INFO: `/v3/projects/9ef278c5009947c98fffdeac5c734a78` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:00.149046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45780b21-9a9c-454b-8fdc-b342bcc67905 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ef278c5009947c98fffdeac5c734a78)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:00.155104 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45780b21-9a9c-454b-8fdc-b342bcc67905 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:00.217846 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-4d846bb5-7240-4001-8072-79c35d2adbd0 None None] MFA Rules not processed for user `4d04998b636649d98103b1cd5f3a0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:00.220512 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4d846bb5-7240-4001-8072-79c35d2adbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:00.258605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-45780b21-9a9c-454b-8fdc-b342bcc67905 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ef278c5009947c98fffdeac5c734a78', 'request_id': 'req-45780b21-9a9c-454b-8fdc-b342bcc67905'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:00.270789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-45780b21-9a9c-454b-8fdc-b342bcc67905 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ef278c5009947c98fffdeac5c734a78', 'request_id': 'req-45780b21-9a9c-454b-8fdc-b342bcc67905'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:00.282998 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:00.283247 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-45780b21-9a9c-454b-8fdc-b342bcc67905 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ef278c5009947c98fffdeac5c734a78', 'request_id': 'req-45780b21-9a9c-454b-8fdc-b342bcc67905'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:00.305889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-45780b21-9a9c-454b-8fdc-b342bcc67905 admin admin] The token cache is being invalidate because project 9ef278c5009947c98fffdeac5c734a78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:00.308305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-45780b21-9a9c-454b-8fdc-b342bcc67905 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:00.309468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-45780b21-9a9c-454b-8fdc-b342bcc67905 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:00.311389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 871/1745] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:00 2022] DELETE /identity/v3/projects/9ef278c5009947c98fffdeac5c734a78 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:00.323547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45780b21-9a9c-454b-8fdc-b342bcc67905 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:00.327421 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-45780b21-9a9c-454b-8fdc-b342bcc67905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:00.333954 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 875/1746] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jul 21 05:44:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:00.533541 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45780b21-9a9c-454b-8fdc-b342bcc67905 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:00.544364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291bc7a5-2a94-4447-862b-511111b4a279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:00.548099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bc7a5-2a94-4447-862b-511111b4a279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:00.549148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bc7a5-2a94-4447-862b-511111b4a279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:00.549428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291bc7a5-2a94-4447-862b-511111b4a279 admin admin] PATH_INFO: `/v3/users/8cf9ca7b97f04558a2a9116041d9dfc4` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:00.555309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291bc7a5-2a94-4447-862b-511111b4a279 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cf9ca7b97f04558a2a9116041d9dfc4)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:00.563449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291bc7a5-2a94-4447-862b-511111b4a279 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:00.616609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d846bb5-7240-4001-8072-79c35d2adbd0 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:00.618301 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4d846bb5-7240-4001-8072-79c35d2adbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:00.666041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:00.777438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-291bc7a5-2a94-4447-862b-511111b4a279 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cf9ca7b97f04558a2a9116041d9dfc4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:00.833469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-291bc7a5-2a94-4447-862b-511111b4a279 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cf9ca7b97f04558a2a9116041d9dfc4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:00.864036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-291bc7a5-2a94-4447-862b-511111b4a279 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cf9ca7b97f04558a2a9116041d9dfc4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:00.865091 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-291bc7a5-2a94-4447-862b-511111b4a279 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cf9ca7b97f04558a2a9116041d9dfc4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:00.867622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-291bc7a5-2a94-4447-862b-511111b4a279 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cf9ca7b97f04558a2a9116041d9dfc4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:00.909449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 872/1747] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:00 2022] DELETE /identity/v3/users/8cf9ca7b97f04558a2a9116041d9dfc4 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:00.914444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8be86c-5140-4e4e-aa7f-1d3eb57a8e75 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:00.917002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8be86c-5140-4e4e-aa7f-1d3eb57a8e75 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:00.918829 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8be86c-5140-4e4e-aa7f-1d3eb57a8e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:00.962532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-1a8be86c-5140-4e4e-aa7f-1d3eb57a8e75 None None] MFA Rules not processed for user `e07508a9fa9146528620b7f8abec8549`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:00.964837 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1a8be86c-5140-4e4e-aa7f-1d3eb57a8e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:01.008354 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d846bb5-7240-4001-8072-79c35d2adbd0 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:01.111174 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 873/1748] 10.222.0.47 () {58 vars in 1015 bytes} [Thu Jul 21 05:44:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:01.158267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca167b2-4fd9-4adf-acff-121f9ae0f5b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61b61188b5b94a51ba1ae1a88b78b89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:01.158836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca167b2-4fd9-4adf-acff-121f9ae0f5b2 service swift] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:01.159284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca167b2-4fd9-4adf-acff-121f9ae0f5b2 service swift] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:01.159568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca167b2-4fd9-4adf-acff-121f9ae0f5b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:01.164882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-bca167b2-4fd9-4adf-acff-121f9ae0f5b2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:01.212293 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8be86c-5140-4e4e-aa7f-1d3eb57a8e75 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:01.213772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1a8be86c-5140-4e4e-aa7f-1d3eb57a8e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:01.278537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca167b2-4fd9-4adf-acff-121f9ae0f5b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:01.280494 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca167b2-4fd9-4adf-acff-121f9ae0f5b2 service swift] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:01.310044 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 876/1749] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jul 21 05:44:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 694 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 05:44:01.311036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca167b2-4fd9-4adf-acff-121f9ae0f5b2 service swift] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:01.312598 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-bca167b2-4fd9-4adf-acff-121f9ae0f5b2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:01.349251 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca167b2-4fd9-4adf-acff-121f9ae0f5b2 service swift] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:01.366698 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d1a40-2147-4197-b082-dca2ef85b69a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:01.367415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d1a40-2147-4197-b082-dca2ef85b69a service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:01.367702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d1a40-2147-4197-b082-dca2ef85b69a service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:01.368041 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d1a40-2147-4197-b082-dca2ef85b69a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:01.380078 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:01.382499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8be86c-5140-4e4e-aa7f-1d3eb57a8e75 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:01.398526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892747e-c7bf-47cf-9226-6b0c8d7a98f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:01.398857 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d1a40-2147-4197-b082-dca2ef85b69a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:01.399469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892747e-c7bf-47cf-9226-6b0c8d7a98f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:01.399955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892747e-c7bf-47cf-9226-6b0c8d7a98f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:01.400455 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892747e-c7bf-47cf-9226-6b0c8d7a98f3 admin admin] PATH_INFO: `/v3/users/7e08156ce92b49df803ea339b6b4c786` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:01.400567 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d1a40-2147-4197-b082-dca2ef85b69a service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:01.401951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892747e-c7bf-47cf-9226-6b0c8d7a98f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e08156ce92b49df803ea339b6b4c786)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:01.404277 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892747e-c7bf-47cf-9226-6b0c8d7a98f3 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:01.423667 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 877/1750] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:01.525008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0892747e-c7bf-47cf-9226-6b0c8d7a98f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e08156ce92b49df803ea339b6b4c786'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:01.539753 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:01.570867 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0892747e-c7bf-47cf-9226-6b0c8d7a98f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e08156ce92b49df803ea339b6b4c786'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:01.608968 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0892747e-c7bf-47cf-9226-6b0c8d7a98f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e08156ce92b49df803ea339b6b4c786'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:01.615143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0892747e-c7bf-47cf-9226-6b0c8d7a98f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e08156ce92b49df803ea339b6b4c786'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:01.616490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0892747e-c7bf-47cf-9226-6b0c8d7a98f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e08156ce92b49df803ea339b6b4c786'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:01.663703 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 874/1751] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:01 2022] DELETE /identity/v3/users/7e08156ce92b49df803ea339b6b4c786 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:01.665871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da4fd0-bd7b-458b-94aa-013982bb3d31 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:01.666171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da4fd0-bd7b-458b-94aa-013982bb3d31 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:01.666450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da4fd0-bd7b-458b-94aa-013982bb3d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:01.716386 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-d1da4fd0-bd7b-458b-94aa-013982bb3d31 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:01.717719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d1da4fd0-bd7b-458b-94aa-013982bb3d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:02.070535 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 878/1752] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:02.141713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892747e-c7bf-47cf-9226-6b0c8d7a98f3 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:02.143466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-0892747e-c7bf-47cf-9226-6b0c8d7a98f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:02.356814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892747e-c7bf-47cf-9226-6b0c8d7a98f3 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:02.379162 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313de36d-87fb-44f7-9dc7-b7450ae97d73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:02.379856 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313de36d-87fb-44f7-9dc7-b7450ae97d73 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:02.380142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313de36d-87fb-44f7-9dc7-b7450ae97d73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:02.380422 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313de36d-87fb-44f7-9dc7-b7450ae97d73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:02.408853 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313de36d-87fb-44f7-9dc7-b7450ae97d73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:02.410707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313de36d-87fb-44f7-9dc7-b7450ae97d73 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:02.451442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 875/1753] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:03.045422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da4fd0-bd7b-458b-94aa-013982bb3d31 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:03.046964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d1da4fd0-bd7b-458b-94aa-013982bb3d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:03.071953 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da4fd0-bd7b-458b-94aa-013982bb3d31 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:03.077058 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2af81-fe1b-4c07-9b1a-4158cbd0d8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:03.077966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2af81-fe1b-4c07-9b1a-4158cbd0d8ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:03.078304 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2af81-fe1b-4c07-9b1a-4158cbd0d8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:03.078587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2af81-fe1b-4c07-9b1a-4158cbd0d8ac admin admin] PATH_INFO: `/v3/projects/ca7f43c19b8443f78d3cce5d5cbba66d` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:03.080015 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2af81-fe1b-4c07-9b1a-4158cbd0d8ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca7f43c19b8443f78d3cce5d5cbba66d)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:03.082229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2af81-fe1b-4c07-9b1a-4158cbd0d8ac admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:03.145903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-80c2af81-fe1b-4c07-9b1a-4158cbd0d8ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca7f43c19b8443f78d3cce5d5cbba66d', 'request_id': 'req-80c2af81-fe1b-4c07-9b1a-4158cbd0d8ac'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:03.153933 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-80c2af81-fe1b-4c07-9b1a-4158cbd0d8ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca7f43c19b8443f78d3cce5d5cbba66d', 'request_id': 'req-80c2af81-fe1b-4c07-9b1a-4158cbd0d8ac'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:03.163049 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-80c2af81-fe1b-4c07-9b1a-4158cbd0d8ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca7f43c19b8443f78d3cce5d5cbba66d', 'request_id': 'req-80c2af81-fe1b-4c07-9b1a-4158cbd0d8ac'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:03.189253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-80c2af81-fe1b-4c07-9b1a-4158cbd0d8ac admin admin] The token cache is being invalidate because project ca7f43c19b8443f78d3cce5d5cbba66d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:03.189628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-80c2af81-fe1b-4c07-9b1a-4158cbd0d8ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:03.190426 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-80c2af81-fe1b-4c07-9b1a-4158cbd0d8ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:03.192777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 879/1754] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:03 2022] DELETE /identity/v3/projects/ca7f43c19b8443f78d3cce5d5cbba66d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:03.198360 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a448204-0cf0-4c08-a2c4-8a932ae29a22 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:03.199800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a448204-0cf0-4c08-a2c4-8a932ae29a22 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:03.200138 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a448204-0cf0-4c08-a2c4-8a932ae29a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:03.250144 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-3a448204-0cf0-4c08-a2c4-8a932ae29a22 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:03.252646 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-3a448204-0cf0-4c08-a2c4-8a932ae29a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:03.355569 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 876/1755] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:03.356862 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b162130-1345-4256-970f-818668956265 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:03.357144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b162130-1345-4256-970f-818668956265 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:03.357472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b162130-1345-4256-970f-818668956265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:03.378540 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-7b162130-1345-4256-970f-818668956265 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:03.380111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7b162130-1345-4256-970f-818668956265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:03.395542 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a448204-0cf0-4c08-a2c4-8a932ae29a22 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:03.403588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 880/1756] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:03.409592 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a448204-0cf0-4c08-a2c4-8a932ae29a22 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:03.416683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cf809-3d7d-4da4-842a-9bde295681bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:03.418214 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cf809-3d7d-4da4-842a-9bde295681bc admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:03.418726 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cf809-3d7d-4da4-842a-9bde295681bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:03.419235 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cf809-3d7d-4da4-842a-9bde295681bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:03.421228 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3cf809-3d7d-4da4-842a-9bde295681bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:44:03.422649 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3cf809-3d7d-4da4-842a-9bde295681bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:03.424620 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3cf809-3d7d-4da4-842a-9bde295681bc admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:03.454360 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 877/1757] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:44:03.462280 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b162130-1345-4256-970f-818668956265 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:03.480929 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b162130-1345-4256-970f-818668956265 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:03.487491 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62dcb83-96b7-4169-ac9a-c66261090158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:03.488060 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62dcb83-96b7-4169-ac9a-c66261090158 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:03.488448 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62dcb83-96b7-4169-ac9a-c66261090158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:03.488810 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62dcb83-96b7-4169-ac9a-c66261090158 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:03.489683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62dcb83-96b7-4169-ac9a-c66261090158 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:03.492507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62dcb83-96b7-4169-ac9a-c66261090158 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:03.579269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cf809-3d7d-4da4-842a-9bde295681bc admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:03.590937 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 881/1758] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:03 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:03.600932 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cf809-3d7d-4da4-842a-9bde295681bc admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:03.857671 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461a2aa-1c1f-4e15-b720-d6855e466fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:03.859509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461a2aa-1c1f-4e15-b720-d6855e466fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:03.859988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461a2aa-1c1f-4e15-b720-d6855e466fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:03.860450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461a2aa-1c1f-4e15-b720-d6855e466fa0 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:03.866566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461a2aa-1c1f-4e15-b720-d6855e466fa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:03.873586 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461a2aa-1c1f-4e15-b720-d6855e466fa0 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:03.903434 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:03.936426 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 878/1759] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:03 2022] POST /identity/v3/users => generated 293 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:03.952269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62dcb83-96b7-4169-ac9a-c66261090158 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:03.964612 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62dcb83-96b7-4169-ac9a-c66261090158 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:03.981832 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e417aea-e479-4fc4-bdd5-9a6df05cf9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:03.982978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e417aea-e479-4fc4-bdd5-9a6df05cf9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:03.983462 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e417aea-e479-4fc4-bdd5-9a6df05cf9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:03.983898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e417aea-e479-4fc4-bdd5-9a6df05cf9b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:03.984806 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e417aea-e479-4fc4-bdd5-9a6df05cf9b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:03.986628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e417aea-e479-4fc4-bdd5-9a6df05cf9b9 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:04.019677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461a2aa-1c1f-4e15-b720-d6855e466fa0 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:04.027258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 882/1760] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:04.031595 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461a2aa-1c1f-4e15-b720-d6855e466fa0 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:04.046691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ec9e5-8c45-4e26-b749-5ad4432a9049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:04.047615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ec9e5-8c45-4e26-b749-5ad4432a9049 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:04.048085 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ec9e5-8c45-4e26-b749-5ad4432a9049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:04.048520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ec9e5-8c45-4e26-b749-5ad4432a9049 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:04.049419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ec9e5-8c45-4e26-b749-5ad4432a9049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:04.051269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ec9e5-8c45-4e26-b749-5ad4432a9049 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:04.076270 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 879/1761] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:04 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:04.079542 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e417aea-e479-4fc4-bdd5-9a6df05cf9b9 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:04.094317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e417aea-e479-4fc4-bdd5-9a6df05cf9b9 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:04.111370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03a492-501d-4337-bcea-beac28e9e446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:04.113082 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03a492-501d-4337-bcea-beac28e9e446 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:04.113671 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03a492-501d-4337-bcea-beac28e9e446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:04.114109 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03a492-501d-4337-bcea-beac28e9e446 admin admin] PATH_INFO: `/v3/projects/f49879fe7ffd4c7e8c780ae5f39e1008/users/bf00b689dcda4293810e86dc97bf5d8b/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:04.133494 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03a492-501d-4337-bcea-beac28e9e446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f49879fe7ffd4c7e8c780ae5f39e1008, user_id=bf00b689dcda4293810e86dc97bf5d8b, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:04.138939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03a492-501d-4337-bcea-beac28e9e446 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:04.164859 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 883/1762] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:04 2022] PUT /identity/v3/projects/f49879fe7ffd4c7e8c780ae5f39e1008/users/bf00b689dcda4293810e86dc97bf5d8b/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:04.179532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942afe1-09af-4e08-9b59-a3ac0870a93c None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:04.180138 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942afe1-09af-4e08-9b59-a3ac0870a93c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:04.180578 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942afe1-09af-4e08-9b59-a3ac0870a93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:04.248060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-e942afe1-09af-4e08-9b59-a3ac0870a93c None None] MFA Rules not processed for user `bf00b689dcda4293810e86dc97bf5d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:04.249351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e942afe1-09af-4e08-9b59-a3ac0870a93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:04.325143 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03a492-501d-4337-bcea-beac28e9e446 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:04.327499 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0f03a492-501d-4337-bcea-beac28e9e446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:04.366805 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:04.461612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 880/1763] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:04.466507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942afe1-09af-4e08-9b59-a3ac0870a93c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:04.515623 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03a492-501d-4337-bcea-beac28e9e446 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:04.531312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf39d9-a30a-4127-9b9f-37efe70950c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:04.532340 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf39d9-a30a-4127-9b9f-37efe70950c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:04.532630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf39d9-a30a-4127-9b9f-37efe70950c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:04.532987 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf39d9-a30a-4127-9b9f-37efe70950c2 admin admin] PATH_INFO: `/v3/projects/56e1c60a9af04d57a84e220a9add1553` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:04.534408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf39d9-a30a-4127-9b9f-37efe70950c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56e1c60a9af04d57a84e220a9add1553)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:04.542670 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf39d9-a30a-4127-9b9f-37efe70950c2 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:04.650408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-53cf39d9-a30a-4127-9b9f-37efe70950c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56e1c60a9af04d57a84e220a9add1553', 'request_id': 'req-53cf39d9-a30a-4127-9b9f-37efe70950c2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:04.654054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942afe1-09af-4e08-9b59-a3ac0870a93c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:04.669786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-53cf39d9-a30a-4127-9b9f-37efe70950c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56e1c60a9af04d57a84e220a9add1553', 'request_id': 'req-53cf39d9-a30a-4127-9b9f-37efe70950c2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:04.687326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-53cf39d9-a30a-4127-9b9f-37efe70950c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56e1c60a9af04d57a84e220a9add1553', 'request_id': 'req-53cf39d9-a30a-4127-9b9f-37efe70950c2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:04.725585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-53cf39d9-a30a-4127-9b9f-37efe70950c2 admin admin] The token cache is being invalidate because project 56e1c60a9af04d57a84e220a9add1553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:04.726387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-53cf39d9-a30a-4127-9b9f-37efe70950c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:04.726739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-53cf39d9-a30a-4127-9b9f-37efe70950c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:04.732825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 884/1764] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:04 2022] DELETE /identity/v3/projects/56e1c60a9af04d57a84e220a9add1553 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:04.734807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310259c-2ff8-432a-abe2-1f3f02034d9f None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:04.734807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310259c-2ff8-432a-abe2-1f3f02034d9f None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:04.735136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310259c-2ff8-432a-abe2-1f3f02034d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:04.758233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547d366-7cbf-4224-b9ce-a5373622de52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:04.759747 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547d366-7cbf-4224-b9ce-a5373622de52 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:04.760134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547d366-7cbf-4224-b9ce-a5373622de52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:04.760506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547d366-7cbf-4224-b9ce-a5373622de52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:04.761451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547d366-7cbf-4224-b9ce-a5373622de52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:04.763704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547d366-7cbf-4224-b9ce-a5373622de52 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:04.764405 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-7310259c-2ff8-432a-abe2-1f3f02034d9f None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:04.766572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7310259c-2ff8-432a-abe2-1f3f02034d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:04.780309 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 885/1765] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:44:04.789060 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35c12d-a466-4689-82ee-04e69bdf4d64 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:04.789345 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35c12d-a466-4689-82ee-04e69bdf4d64 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:04.789945 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35c12d-a466-4689-82ee-04e69bdf4d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:04.792836 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:04.814362 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 881/1766] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:04 2022] POST /identity/v3/projects => generated 385 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:04.823609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547d366-7cbf-4224-b9ce-a5373622de52 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:04.823830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-5e35c12d-a466-4689-82ee-04e69bdf4d64 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:04.824810 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5e35c12d-a466-4689-82ee-04e69bdf4d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:04.827540 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6547d366-7cbf-4224-b9ce-a5373622de52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:04.968905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 886/1767] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:04.970117 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35c12d-a466-4689-82ee-04e69bdf4d64 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:04.986302 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35c12d-a466-4689-82ee-04e69bdf4d64 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:04.990939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547d366-7cbf-4224-b9ce-a5373622de52 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:04.992944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411343d4-0ea2-4642-9bd8-eb0c6499cf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:04.994267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411343d4-0ea2-4642-9bd8-eb0c6499cf0b admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:04.994944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411343d4-0ea2-4642-9bd8-eb0c6499cf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:04.995482 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411343d4-0ea2-4642-9bd8-eb0c6499cf0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:04.996756 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411343d4-0ea2-4642-9bd8-eb0c6499cf0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:44:04.997971 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411343d4-0ea2-4642-9bd8-eb0c6499cf0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:04.999900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411343d4-0ea2-4642-9bd8-eb0c6499cf0b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:05.010145 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82453d5d-00a1-4a7b-8e08-828a766eb378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:05.010963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82453d5d-00a1-4a7b-8e08-828a766eb378 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:05.011613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82453d5d-00a1-4a7b-8e08-828a766eb378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:05.012220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82453d5d-00a1-4a7b-8e08-828a766eb378 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:05.013185 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82453d5d-00a1-4a7b-8e08-828a766eb378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:05.015573 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82453d5d-00a1-4a7b-8e08-828a766eb378 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:05.018634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 887/1768] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:44:05.022553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411343d4-0ea2-4642-9bd8-eb0c6499cf0b admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:05.037613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411343d4-0ea2-4642-9bd8-eb0c6499cf0b admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:05.043554 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7238965-bdb1-4a4d-acf0-1ec1333d469e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:05.044463 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7238965-bdb1-4a4d-acf0-1ec1333d469e admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:05.045031 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7238965-bdb1-4a4d-acf0-1ec1333d469e admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:05.045486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7238965-bdb1-4a4d-acf0-1ec1333d469e admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:05.046443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7238965-bdb1-4a4d-acf0-1ec1333d469e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:05.048570 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7238965-bdb1-4a4d-acf0-1ec1333d469e admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:05.051678 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 882/1769] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:04 2022] POST /identity/v3/users => generated 299 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:05.056182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82453d5d-00a1-4a7b-8e08-828a766eb378 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:05.071207 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82453d5d-00a1-4a7b-8e08-828a766eb378 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:05.083256 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1102e4-5315-40bd-84db-02b2eee1d300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:05.084542 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1102e4-5315-40bd-84db-02b2eee1d300 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:05.085003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1102e4-5315-40bd-84db-02b2eee1d300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:05.085380 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1102e4-5315-40bd-84db-02b2eee1d300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:05.086946 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1102e4-5315-40bd-84db-02b2eee1d300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:05.088754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1102e4-5315-40bd-84db-02b2eee1d300 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:05.105323 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 888/1770] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:05 2022] POST /identity/v3/projects => generated 379 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:44:05.108994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 883/1771] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:44:05.112709 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7238965-bdb1-4a4d-acf0-1ec1333d469e admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:05.113671 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1102e4-5315-40bd-84db-02b2eee1d300 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:05.118176 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:05.126868 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1102e4-5315-40bd-84db-02b2eee1d300 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:05.238171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7238965-bdb1-4a4d-acf0-1ec1333d469e admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:05.250898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012249c-479f-4b60-8277-0828f07e9d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:05.251646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012249c-479f-4b60-8277-0828f07e9d9d admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:05.251944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012249c-479f-4b60-8277-0828f07e9d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:05.252225 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012249c-479f-4b60-8277-0828f07e9d9d admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:05.252938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012249c-479f-4b60-8277-0828f07e9d9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:05.254911 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012249c-479f-4b60-8277-0828f07e9d9d admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:05.261904 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12128c-1b54-45a2-a101-2419be818d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:05.263005 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12128c-1b54-45a2-a101-2419be818d5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:05.263309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12128c-1b54-45a2-a101-2419be818d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:05.263576 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12128c-1b54-45a2-a101-2419be818d5a admin admin] PATH_INFO: `/v3/projects/3aa7d76c23ae4462a0e997c60bd46a07/users/477579e3cf10494ab0bbbcb9d45814f4/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:05.286875 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d12128c-1b54-45a2-a101-2419be818d5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aa7d76c23ae4462a0e997c60bd46a07, user_id=477579e3cf10494ab0bbbcb9d45814f4, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:05.289260 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 889/1772] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:05 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:05.293155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012249c-479f-4b60-8277-0828f07e9d9d admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:05.293475 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d12128c-1b54-45a2-a101-2419be818d5a admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:05.315728 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 884/1773] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:05 2022] PUT /identity/v3/projects/3aa7d76c23ae4462a0e997c60bd46a07/users/477579e3cf10494ab0bbbcb9d45814f4/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:05.319491 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12128c-1b54-45a2-a101-2419be818d5a admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:05.324630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012249c-479f-4b60-8277-0828f07e9d9d admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:05.490186 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12128c-1b54-45a2-a101-2419be818d5a admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:05.495253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f185a69-c40b-4993-9ecc-b98d1bbe6f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:05.496056 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185a69-c40b-4993-9ecc-b98d1bbe6f1e admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:05.496344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185a69-c40b-4993-9ecc-b98d1bbe6f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:05.496834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f185a69-c40b-4993-9ecc-b98d1bbe6f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:05.497703 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f185a69-c40b-4993-9ecc-b98d1bbe6f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:05.499496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f185a69-c40b-4993-9ecc-b98d1bbe6f1e admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:05.502564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:05.515588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd46676-3109-4fba-a3a5-8368908f75be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:05.515768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 885/1774] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:44:05.517323 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd46676-3109-4fba-a3a5-8368908f75be admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:05.517682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd46676-3109-4fba-a3a5-8368908f75be admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:05.517930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8335a0-949f-496d-90d5-82c0e2e19225 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:05.518052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd46676-3109-4fba-a3a5-8368908f75be admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:05.518344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8335a0-949f-496d-90d5-82c0e2e19225 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:05.518697 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8335a0-949f-496d-90d5-82c0e2e19225 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:05.520651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd46676-3109-4fba-a3a5-8368908f75be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:05.521678 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 886/1775] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jul 21 05:44:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:44:05.522786 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8335a0-949f-496d-90d5-82c0e2e19225 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:05.528980 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd46676-3109-4fba-a3a5-8368908f75be admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:05.538491 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8335a0-949f-496d-90d5-82c0e2e19225 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:05.544588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46690067-311f-40dc-9b29-f3c0a23b0b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:05.546160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46690067-311f-40dc-9b29-f3c0a23b0b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:05.546514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46690067-311f-40dc-9b29-f3c0a23b0b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:05.546865 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46690067-311f-40dc-9b29-f3c0a23b0b6d admin admin] PATH_INFO: `/v3/projects/3aa7d76c23ae4462a0e997c60bd46a07/users/477579e3cf10494ab0bbbcb9d45814f4/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:05.549669 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46690067-311f-40dc-9b29-f3c0a23b0b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aa7d76c23ae4462a0e997c60bd46a07, user_id=477579e3cf10494ab0bbbcb9d45814f4, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:05.552495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46690067-311f-40dc-9b29-f3c0a23b0b6d admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:05.563263 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 890/1776] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:44:05.563263 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c5b1d-8315-47cd-a926-cc62b6bafbcb None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:05.563263 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c5b1d-8315-47cd-a926-cc62b6bafbcb None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:05.563263 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c5b1d-8315-47cd-a926-cc62b6bafbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:05.577455 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 887/1777] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:05 2022] PUT /identity/v3/projects/3aa7d76c23ae4462a0e997c60bd46a07/users/477579e3cf10494ab0bbbcb9d45814f4/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:05.578578 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46690067-311f-40dc-9b29-f3c0a23b0b6d admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:05.581582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-bf3c5b1d-8315-47cd-a926-cc62b6bafbcb None None] MFA Rules not processed for user `928d014648cd41d7b58e7e01845eb748`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:05.583547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-bf3c5b1d-8315-47cd-a926-cc62b6bafbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:05.669020 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46690067-311f-40dc-9b29-f3c0a23b0b6d admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:05.674666 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9a58c-490d-41a4-b69c-f58ce460b12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:05.674891 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 891/1778] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jul 21 05:44:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:05.676138 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b70259-9b06-45a6-a978-5754814e7e80 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:05.676320 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9a58c-490d-41a4-b69c-f58ce460b12b admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:05.676422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b70259-9b06-45a6-a978-5754814e7e80 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:05.676692 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b70259-9b06-45a6-a978-5754814e7e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:05.677389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9a58c-490d-41a4-b69c-f58ce460b12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:05.678294 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9a58c-490d-41a4-b69c-f58ce460b12b admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:05.679239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9a58c-490d-41a4-b69c-f58ce460b12b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:05.682632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e9a58c-490d-41a4-b69c-f58ce460b12b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:05.707391 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 888/1779] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:05.707391 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9a58c-490d-41a4-b69c-f58ce460b12b admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:05.707391 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-80e9a58c-490d-41a4-b69c-f58ce460b12b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:05.714506 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-11b70259-9b06-45a6-a978-5754814e7e80 None None] MFA Rules not processed for user `477579e3cf10494ab0bbbcb9d45814f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:05.716385 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-11b70259-9b06-45a6-a978-5754814e7e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:05.726478 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9a58c-490d-41a4-b69c-f58ce460b12b admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:05.728415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:05.735825 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd516089-af56-4a5d-8674-629968d5a05a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:05.736489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd516089-af56-4a5d-8674-629968d5a05a service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:05.736780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd516089-af56-4a5d-8674-629968d5a05a service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:05.737052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd516089-af56-4a5d-8674-629968d5a05a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:05.749328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd516089-af56-4a5d-8674-629968d5a05a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:05.750945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd516089-af56-4a5d-8674-629968d5a05a service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:05.763041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 889/1780] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:05.763973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd516089-af56-4a5d-8674-629968d5a05a service nova] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:05.776070 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd516089-af56-4a5d-8674-629968d5a05a service nova] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:05.782156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9a4af-93e4-4eb8-8e5b-f5a610378dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:05.783208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9a4af-93e4-4eb8-8e5b-f5a610378dac admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:05.783567 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9a4af-93e4-4eb8-8e5b-f5a610378dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:05.783912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9a4af-93e4-4eb8-8e5b-f5a610378dac admin admin] PATH_INFO: `/v3/projects/61953f626c684582a7642fa6b9c0fc0c/users/22b9864177804ce590571c8e8ee412dc/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:05.809284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9a4af-93e4-4eb8-8e5b-f5a610378dac admin admin] RBAC: Authorizing `identity:create_grant(project_id=61953f626c684582a7642fa6b9c0fc0c, user_id=22b9864177804ce590571c8e8ee412dc, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:05.812761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9a4af-93e4-4eb8-8e5b-f5a610378dac admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:05.822359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 892/1781] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:05.829602 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 890/1782] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:05 2022] PUT /identity/v3/projects/61953f626c684582a7642fa6b9c0fc0c/users/22b9864177804ce590571c8e8ee412dc/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:05.836675 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b70259-9b06-45a6-a978-5754814e7e80 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:05.839515 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-11b70259-9b06-45a6-a978-5754814e7e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:05.841214 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e2281-5f8d-4982-9fa3-29774ef4c797 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:05.841505 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e2281-5f8d-4982-9fa3-29774ef4c797 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:05.841784 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e2281-5f8d-4982-9fa3-29774ef4c797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:05.870589 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-1e0e2281-5f8d-4982-9fa3-29774ef4c797 None None] MFA Rules not processed for user `22b9864177804ce590571c8e8ee412dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:05.871860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1e0e2281-5f8d-4982-9fa3-29774ef4c797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:05.900850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b70259-9b06-45a6-a978-5754814e7e80 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:05.943755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 891/1783] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:05.948093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e2281-5f8d-4982-9fa3-29774ef4c797 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:05.950895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b225c7-3481-4a56-a53a-7930d4a1c8b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:05.951635 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b225c7-3481-4a56-a53a-7930d4a1c8b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:05.951921 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b225c7-3481-4a56-a53a-7930d4a1c8b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:05.952200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b225c7-3481-4a56-a53a-7930d4a1c8b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:05.985082 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:06.014507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b225c7-3481-4a56-a53a-7930d4a1c8b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:06.016081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b225c7-3481-4a56-a53a-7930d4a1c8b2 service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:06.016779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e2281-5f8d-4982-9fa3-29774ef4c797 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:06.021591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3256be-b9fc-47d8-9119-f6395e627d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:06.022455 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3256be-b9fc-47d8-9119-f6395e627d81 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:06.022879 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3256be-b9fc-47d8-9119-f6395e627d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:06.023216 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3256be-b9fc-47d8-9119-f6395e627d81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:06.025476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3256be-b9fc-47d8-9119-f6395e627d81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:06.027481 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3256be-b9fc-47d8-9119-f6395e627d81 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:06.028463 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 893/1784] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:06.059898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 892/1785] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:05 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:44:06.067916 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b225c7-3481-4a56-a53a-7930d4a1c8b2 service cinder] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:06.073113 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3256be-b9fc-47d8-9119-f6395e627d81 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:06.081185 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b225c7-3481-4a56-a53a-7930d4a1c8b2 service cinder] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:06.089723 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:06.150164 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3256be-b9fc-47d8-9119-f6395e627d81 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:06.153141 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d6143-af74-43f3-ab64-993bde9f6d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:06.153914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d6143-af74-43f3-ab64-993bde9f6d8b admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:06.154207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d6143-af74-43f3-ab64-993bde9f6d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:06.154575 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d6143-af74-43f3-ab64-993bde9f6d8b admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:06.155460 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d6143-af74-43f3-ab64-993bde9f6d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:06.156668 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6723a2-5832-4066-9657-e37c0cfc9e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:06.157423 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d6143-af74-43f3-ab64-993bde9f6d8b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:06.157885 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6723a2-5832-4066-9657-e37c0cfc9e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:06.158309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6723a2-5832-4066-9657-e37c0cfc9e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:06.158723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6723a2-5832-4066-9657-e37c0cfc9e56 admin admin] PATH_INFO: `/v3/users/bf00b689dcda4293810e86dc97bf5d8b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:06.160592 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6723a2-5832-4066-9657-e37c0cfc9e56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf00b689dcda4293810e86dc97bf5d8b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:06.162730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6723a2-5832-4066-9657-e37c0cfc9e56 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:06.187080 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 894/1786] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:06 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:06.189306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d6143-af74-43f3-ab64-993bde9f6d8b admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:06.201089 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d6143-af74-43f3-ab64-993bde9f6d8b admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:06.206435 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3807e0-5b7f-45c4-8bc5-33a84d5e7923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:06.207148 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3807e0-5b7f-45c4-8bc5-33a84d5e7923 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:06.207432 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3807e0-5b7f-45c4-8bc5-33a84d5e7923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:06.207711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3807e0-5b7f-45c4-8bc5-33a84d5e7923 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:06.208436 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3807e0-5b7f-45c4-8bc5-33a84d5e7923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:06.210066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3807e0-5b7f-45c4-8bc5-33a84d5e7923 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:06.221983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 895/1787] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:44:06.224682 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-1b6723a2-5832-4066-9657-e37c0cfc9e56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf00b689dcda4293810e86dc97bf5d8b'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:06.225987 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3807e0-5b7f-45c4-8bc5-33a84d5e7923 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:06.240271 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3807e0-5b7f-45c4-8bc5-33a84d5e7923 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:06.247510 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-1b6723a2-5832-4066-9657-e37c0cfc9e56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf00b689dcda4293810e86dc97bf5d8b'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:06.248859 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c6caf-2553-4ffb-ab81-9b0018254704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:06.249467 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c6caf-2553-4ffb-ab81-9b0018254704 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:06.249761 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c6caf-2553-4ffb-ab81-9b0018254704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:06.250054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c6caf-2553-4ffb-ab81-9b0018254704 admin admin] PATH_INFO: `/v3/projects/ed24d495cd4543bba2519c63dfb681ea/users/199c008f282c400ea18e5ab2d434eacb/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:06.255239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-1b6723a2-5832-4066-9657-e37c0cfc9e56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf00b689dcda4293810e86dc97bf5d8b'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:06.255973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-1b6723a2-5832-4066-9657-e37c0cfc9e56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf00b689dcda4293810e86dc97bf5d8b'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:06.256599 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-1b6723a2-5832-4066-9657-e37c0cfc9e56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf00b689dcda4293810e86dc97bf5d8b'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:06.266501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c6caf-2553-4ffb-ab81-9b0018254704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed24d495cd4543bba2519c63dfb681ea, user_id=199c008f282c400ea18e5ab2d434eacb, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:06.269339 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c6caf-2553-4ffb-ab81-9b0018254704 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:06.281223 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 893/1788] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:06 2022] DELETE /identity/v3/users/bf00b689dcda4293810e86dc97bf5d8b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:06.284895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 896/1789] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:06 2022] PUT /identity/v3/projects/ed24d495cd4543bba2519c63dfb681ea/users/199c008f282c400ea18e5ab2d434eacb/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:06.286056 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a9980-34e0-453d-acd7-3c9bec0170ff None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:06.286350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a9980-34e0-453d-acd7-3c9bec0170ff None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:06.286631 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a9980-34e0-453d-acd7-3c9bec0170ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:06.289095 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c6caf-2553-4ffb-ab81-9b0018254704 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:06.290751 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0e5c6caf-2553-4ffb-ab81-9b0018254704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:06.304271 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-ac6a9980-34e0-453d-acd7-3c9bec0170ff None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:06.305466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ac6a9980-34e0-453d-acd7-3c9bec0170ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:06.336962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:06.354476 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c6caf-2553-4ffb-ab81-9b0018254704 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:06.365352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 894/1790] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:44:06.375701 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a9980-34e0-453d-acd7-3c9bec0170ff None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:06.377244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ac6a9980-34e0-453d-acd7-3c9bec0170ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:06.409881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5b3e3-c0a3-453e-8b8e-2dc5d012296c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:06.410911 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5b3e3-c0a3-453e-8b8e-2dc5d012296c admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:06.411258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5b3e3-c0a3-453e-8b8e-2dc5d012296c admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:06.411534 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5b3e3-c0a3-453e-8b8e-2dc5d012296c admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:06.412269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5b3e3-c0a3-453e-8b8e-2dc5d012296c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:06.413898 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5b3e3-c0a3-453e-8b8e-2dc5d012296c admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:06.423855 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 897/1791] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:44:06.428023 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5b3e3-c0a3-453e-8b8e-2dc5d012296c admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:06.437788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5b3e3-c0a3-453e-8b8e-2dc5d012296c admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:06.437938 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a9980-34e0-453d-acd7-3c9bec0170ff None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:06.441793 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24521f-f9ff-4c4f-aa4b-8f0853c76e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:06.442455 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24521f-f9ff-4c4f-aa4b-8f0853c76e58 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:06.442738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24521f-f9ff-4c4f-aa4b-8f0853c76e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:06.442869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b60b76-d4d3-4ff5-b030-e2e9555f404f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:06.442971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb24521f-f9ff-4c4f-aa4b-8f0853c76e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:06.443685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b60b76-d4d3-4ff5-b030-e2e9555f404f admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:06.443972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b60b76-d4d3-4ff5-b030-e2e9555f404f admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:06.444251 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b60b76-d4d3-4ff5-b030-e2e9555f404f admin admin] PATH_INFO: `/v3/projects/ed24d495cd4543bba2519c63dfb681ea/users/199c008f282c400ea18e5ab2d434eacb/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:06.446246 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b60b76-d4d3-4ff5-b030-e2e9555f404f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed24d495cd4543bba2519c63dfb681ea, user_id=199c008f282c400ea18e5ab2d434eacb, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:06.449033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b60b76-d4d3-4ff5-b030-e2e9555f404f admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:06.452256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:06.453387 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24521f-f9ff-4c4f-aa4b-8f0853c76e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:06.454881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb24521f-f9ff-4c4f-aa4b-8f0853c76e58 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:06.461189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 898/1792] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:06 2022] PUT /identity/v3/projects/ed24d495cd4543bba2519c63dfb681ea/users/199c008f282c400ea18e5ab2d434eacb/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:06.464747 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 895/1793] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:06.471320 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a534860-9324-4151-bece-a54725eb13a5 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:06.471670 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a534860-9324-4151-bece-a54725eb13a5 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:06.471962 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a534860-9324-4151-bece-a54725eb13a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:06.500805 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-0a534860-9324-4151-bece-a54725eb13a5 None None] MFA Rules not processed for user `199c008f282c400ea18e5ab2d434eacb`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:06.502165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0a534860-9324-4151-bece-a54725eb13a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:06.582193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 899/1794] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:06.594020 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24521f-f9ff-4c4f-aa4b-8f0853c76e58 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:06.595593 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-fb24521f-f9ff-4c4f-aa4b-8f0853c76e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:06.677781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb24521f-f9ff-4c4f-aa4b-8f0853c76e58 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:06.736887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210e522-a54b-4f25-bdc5-c5e7f65948c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:06.737911 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210e522-a54b-4f25-bdc5-c5e7f65948c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:06.737911 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210e522-a54b-4f25-bdc5-c5e7f65948c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:06.738088 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210e522-a54b-4f25-bdc5-c5e7f65948c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:06.755829 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210e522-a54b-4f25-bdc5-c5e7f65948c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:06.757483 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210e522-a54b-4f25-bdc5-c5e7f65948c5 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:06.770785 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a534860-9324-4151-bece-a54725eb13a5 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:06.771674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 896/1795] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:06.774466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0a534860-9324-4151-bece-a54725eb13a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:06.867068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210e522-a54b-4f25-bdc5-c5e7f65948c5 service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:06.875645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210e522-a54b-4f25-bdc5-c5e7f65948c5 service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:06.879391 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526082e3-e7d2-4abf-aaf1-2948a883027a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:06.880049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526082e3-e7d2-4abf-aaf1-2948a883027a service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:06.880336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526082e3-e7d2-4abf-aaf1-2948a883027a service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:06.880718 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526082e3-e7d2-4abf-aaf1-2948a883027a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:06.880985 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a534860-9324-4151-bece-a54725eb13a5 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:06.882505 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-526082e3-e7d2-4abf-aaf1-2948a883027a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:06.885905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d423a30-5cfe-4793-9f87-8f1715fd51aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:06.886890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d423a30-5cfe-4793-9f87-8f1715fd51aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:06.887300 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d423a30-5cfe-4793-9f87-8f1715fd51aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:06.887696 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d423a30-5cfe-4793-9f87-8f1715fd51aa admin admin] PATH_INFO: `/v3/projects/f49879fe7ffd4c7e8c780ae5f39e1008` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:06.889142 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d423a30-5cfe-4793-9f87-8f1715fd51aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f49879fe7ffd4c7e8c780ae5f39e1008)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:06.891209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d423a30-5cfe-4793-9f87-8f1715fd51aa admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:06.927650 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:06.940852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9d423a30-5cfe-4793-9f87-8f1715fd51aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f49879fe7ffd4c7e8c780ae5f39e1008', 'request_id': 'req-9d423a30-5cfe-4793-9f87-8f1715fd51aa'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:06.947445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9d423a30-5cfe-4793-9f87-8f1715fd51aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f49879fe7ffd4c7e8c780ae5f39e1008', 'request_id': 'req-9d423a30-5cfe-4793-9f87-8f1715fd51aa'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:06.947944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526082e3-e7d2-4abf-aaf1-2948a883027a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:06.949414 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526082e3-e7d2-4abf-aaf1-2948a883027a service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:06.952897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9d423a30-5cfe-4793-9f87-8f1715fd51aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f49879fe7ffd4c7e8c780ae5f39e1008', 'request_id': 'req-9d423a30-5cfe-4793-9f87-8f1715fd51aa'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:06.959119 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 897/1796] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:44:06.966849 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9d423a30-5cfe-4793-9f87-8f1715fd51aa admin admin] The token cache is being invalidate because project f49879fe7ffd4c7e8c780ae5f39e1008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:06.967179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9d423a30-5cfe-4793-9f87-8f1715fd51aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:06.967896 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9d423a30-5cfe-4793-9f87-8f1715fd51aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:06.969369 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 900/1797] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:06 2022] DELETE /identity/v3/projects/f49879fe7ffd4c7e8c780ae5f39e1008 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:06.973768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526082e3-e7d2-4abf-aaf1-2948a883027a service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:06.985505 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526082e3-e7d2-4abf-aaf1-2948a883027a service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:07.082221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197f256-a7f7-4ed9-8c84-a90bda26adc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:07.083207 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197f256-a7f7-4ed9-8c84-a90bda26adc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:07.083503 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197f256-a7f7-4ed9-8c84-a90bda26adc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:07.083926 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197f256-a7f7-4ed9-8c84-a90bda26adc4 admin admin] PATH_INFO: `/v3/users/477579e3cf10494ab0bbbcb9d45814f4` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:07.085233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197f256-a7f7-4ed9-8c84-a90bda26adc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=477579e3cf10494ab0bbbcb9d45814f4)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:07.088416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197f256-a7f7-4ed9-8c84-a90bda26adc4 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:07.177643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0197f256-a7f7-4ed9-8c84-a90bda26adc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '477579e3cf10494ab0bbbcb9d45814f4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:07.198794 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0197f256-a7f7-4ed9-8c84-a90bda26adc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '477579e3cf10494ab0bbbcb9d45814f4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:07.205998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0197f256-a7f7-4ed9-8c84-a90bda26adc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '477579e3cf10494ab0bbbcb9d45814f4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:07.206760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0197f256-a7f7-4ed9-8c84-a90bda26adc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '477579e3cf10494ab0bbbcb9d45814f4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:07.207483 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0197f256-a7f7-4ed9-8c84-a90bda26adc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '477579e3cf10494ab0bbbcb9d45814f4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:07.225425 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 898/1798] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:06 2022] DELETE /identity/v3/users/477579e3cf10494ab0bbbcb9d45814f4 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:07.592280 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d423a30-5cfe-4793-9f87-8f1715fd51aa admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:07.593744 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9d423a30-5cfe-4793-9f87-8f1715fd51aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:07.608786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d423a30-5cfe-4793-9f87-8f1715fd51aa admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:07.627108 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:07.698226 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9c821-c282-404e-822a-5bbb48ac7fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:07.699694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9c821-c282-404e-822a-5bbb48ac7fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:07.700032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9c821-c282-404e-822a-5bbb48ac7fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:07.700379 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9c821-c282-404e-822a-5bbb48ac7fc2 admin admin] PATH_INFO: `/v3/projects/3aa7d76c23ae4462a0e997c60bd46a07` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:07.703057 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9c821-c282-404e-822a-5bbb48ac7fc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aa7d76c23ae4462a0e997c60bd46a07)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:07.705304 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9c821-c282-404e-822a-5bbb48ac7fc2 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:07.770025 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-16b9c821-c282-404e-822a-5bbb48ac7fc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aa7d76c23ae4462a0e997c60bd46a07', 'request_id': 'req-16b9c821-c282-404e-822a-5bbb48ac7fc2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:07.777385 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-16b9c821-c282-404e-822a-5bbb48ac7fc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aa7d76c23ae4462a0e997c60bd46a07', 'request_id': 'req-16b9c821-c282-404e-822a-5bbb48ac7fc2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:07.783611 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-16b9c821-c282-404e-822a-5bbb48ac7fc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aa7d76c23ae4462a0e997c60bd46a07', 'request_id': 'req-16b9c821-c282-404e-822a-5bbb48ac7fc2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:07.797773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-16b9c821-c282-404e-822a-5bbb48ac7fc2 admin admin] The token cache is being invalidate because project 3aa7d76c23ae4462a0e997c60bd46a07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:07.798060 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-16b9c821-c282-404e-822a-5bbb48ac7fc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:07.798682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-16b9c821-c282-404e-822a-5bbb48ac7fc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:07.800269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 901/1799] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:07 2022] DELETE /identity/v3/projects/3aa7d76c23ae4462a0e997c60bd46a07 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:07.806621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963051e-adbf-4442-9773-bad6f1a2e383 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:07.807188 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963051e-adbf-4442-9773-bad6f1a2e383 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:07.807282 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963051e-adbf-4442-9773-bad6f1a2e383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:07.842854 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-9963051e-adbf-4442-9773-bad6f1a2e383 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:07.844159 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-9963051e-adbf-4442-9773-bad6f1a2e383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:07.905458 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:07.919303 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 899/1800] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:07.927591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778f3c6-2a3c-4979-9e99-526e2467f288 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:07.927885 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778f3c6-2a3c-4979-9e99-526e2467f288 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:07.928163 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778f3c6-2a3c-4979-9e99-526e2467f288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:07.964757 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-4778f3c6-2a3c-4979-9e99-526e2467f288 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:07.966229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4778f3c6-2a3c-4979-9e99-526e2467f288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:07.974742 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 902/1801] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:44:07.979245 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963051e-adbf-4442-9773-bad6f1a2e383 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:07.996812 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963051e-adbf-4442-9773-bad6f1a2e383 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:08.002552 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97699200-77b0-4988-a2bc-202ee2d6313a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:08.003309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97699200-77b0-4988-a2bc-202ee2d6313a admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:08.004565 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97699200-77b0-4988-a2bc-202ee2d6313a admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:08.004810 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97699200-77b0-4988-a2bc-202ee2d6313a admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:08.005576 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97699200-77b0-4988-a2bc-202ee2d6313a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:44:08.005921 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97699200-77b0-4988-a2bc-202ee2d6313a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:08.011134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97699200-77b0-4988-a2bc-202ee2d6313a admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:08.033967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 900/1802] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:44:08.035622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778f3c6-2a3c-4979-9e99-526e2467f288 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:08.045801 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778f3c6-2a3c-4979-9e99-526e2467f288 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:08.050641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643c7ce-5ebc-41ad-ad59-56529cd0fc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:08.051555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643c7ce-5ebc-41ad-ad59-56529cd0fc70 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:08.051936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643c7ce-5ebc-41ad-ad59-56529cd0fc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:08.052294 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643c7ce-5ebc-41ad-ad59-56529cd0fc70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:08.053012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643c7ce-5ebc-41ad-ad59-56529cd0fc70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:08.055945 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643c7ce-5ebc-41ad-ad59-56529cd0fc70 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:08.103833 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 903/1803] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:08 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:08.105080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97699200-77b0-4988-a2bc-202ee2d6313a admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:08.116486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97699200-77b0-4988-a2bc-202ee2d6313a admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:08.194422 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41468d2-aca2-42fa-ad4e-fc77a8edc499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:08.195343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41468d2-aca2-42fa-ad4e-fc77a8edc499 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:08.195915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41468d2-aca2-42fa-ad4e-fc77a8edc499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:08.196289 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41468d2-aca2-42fa-ad4e-fc77a8edc499 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:08.197290 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41468d2-aca2-42fa-ad4e-fc77a8edc499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:08.199820 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41468d2-aca2-42fa-ad4e-fc77a8edc499 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:08.225418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 901/1804] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:08 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:08.229633 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643c7ce-5ebc-41ad-ad59-56529cd0fc70 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:08.240364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643c7ce-5ebc-41ad-ad59-56529cd0fc70 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:08.244928 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967e175-781d-4959-9cfd-8b224f2182af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:08.245604 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967e175-781d-4959-9cfd-8b224f2182af admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:08.245887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967e175-781d-4959-9cfd-8b224f2182af admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:08.246160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967e175-781d-4959-9cfd-8b224f2182af admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:08.247149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967e175-781d-4959-9cfd-8b224f2182af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:08.248873 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967e175-781d-4959-9cfd-8b224f2182af admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:08.271956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 904/1805] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:08.273507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41468d2-aca2-42fa-ad4e-fc77a8edc499 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:08.285777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41468d2-aca2-42fa-ad4e-fc77a8edc499 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:08.290931 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df337a5-7ccc-4a90-8500-45fbfdeb7f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:08.291642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df337a5-7ccc-4a90-8500-45fbfdeb7f89 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:08.291981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df337a5-7ccc-4a90-8500-45fbfdeb7f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:08.292250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df337a5-7ccc-4a90-8500-45fbfdeb7f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:08.293167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df337a5-7ccc-4a90-8500-45fbfdeb7f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:08.295002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df337a5-7ccc-4a90-8500-45fbfdeb7f89 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:08.310424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 902/1806] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:08.312139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967e175-781d-4959-9cfd-8b224f2182af admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:08.325306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967e175-781d-4959-9cfd-8b224f2182af admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:08.334416 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf50fa1-5d8b-4b89-b058-ec4a7144711e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:08.335824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf50fa1-5d8b-4b89-b058-ec4a7144711e admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:08.336744 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf50fa1-5d8b-4b89-b058-ec4a7144711e admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:08.337522 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf50fa1-5d8b-4b89-b058-ec4a7144711e admin admin] PATH_INFO: `/v3/projects/2caf329bfe474924b19ffbfc46d90205/users/2ad78f8b5fae4017a2b9630b37bb1ebf/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:08.359969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf50fa1-5d8b-4b89-b058-ec4a7144711e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2caf329bfe474924b19ffbfc46d90205, user_id=2ad78f8b5fae4017a2b9630b37bb1ebf, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:08.362825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf50fa1-5d8b-4b89-b058-ec4a7144711e admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:08.408235 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 905/1807] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:08 2022] PUT /identity/v3/projects/2caf329bfe474924b19ffbfc46d90205/users/2ad78f8b5fae4017a2b9630b37bb1ebf/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:08.419462 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc93ab-168e-47ff-8043-efc10d32bb05 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:08.419825 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc93ab-168e-47ff-8043-efc10d32bb05 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:08.420369 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc93ab-168e-47ff-8043-efc10d32bb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:08.454731 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-c9dc93ab-168e-47ff-8043-efc10d32bb05 None None] MFA Rules not processed for user `2ad78f8b5fae4017a2b9630b37bb1ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:08.456657 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c9dc93ab-168e-47ff-8043-efc10d32bb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:08.599949 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 903/1808] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:08.601351 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf50fa1-5d8b-4b89-b058-ec4a7144711e admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:08.700986 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:08.746887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf50fa1-5d8b-4b89-b058-ec4a7144711e admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:08.752132 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a71cea-5a11-412c-9483-bf1c77fa20a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:08.752939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a71cea-5a11-412c-9483-bf1c77fa20a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:08.753318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a71cea-5a11-412c-9483-bf1c77fa20a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:08.753688 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a71cea-5a11-412c-9483-bf1c77fa20a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:08.754899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a71cea-5a11-412c-9483-bf1c77fa20a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:08.757628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a71cea-5a11-412c-9483-bf1c77fa20a0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:08.809226 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 906/1809] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:08 2022] POST /identity/v3/projects => generated 387 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:08.811111 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc93ab-168e-47ff-8043-efc10d32bb05 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:08.832257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc93ab-168e-47ff-8043-efc10d32bb05 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:08.951891 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:08.962495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca278e3f-5aa6-415e-975f-337179d14230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:08.963853 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca278e3f-5aa6-415e-975f-337179d14230 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:08.964418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca278e3f-5aa6-415e-975f-337179d14230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:08.964898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca278e3f-5aa6-415e-975f-337179d14230 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:08.965883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca278e3f-5aa6-415e-975f-337179d14230 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:08.968368 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca278e3f-5aa6-415e-975f-337179d14230 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:09.005244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 904/1810] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:08 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:09.006131 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a71cea-5a11-412c-9483-bf1c77fa20a0 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:09.030436 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a71cea-5a11-412c-9483-bf1c77fa20a0 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:09.035494 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029d14a-b446-4a0e-b984-d3bbd562921d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:09.036431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029d14a-b446-4a0e-b984-d3bbd562921d admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:09.036676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029d14a-b446-4a0e-b984-d3bbd562921d admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:09.037065 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029d14a-b446-4a0e-b984-d3bbd562921d admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:09.037800 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029d14a-b446-4a0e-b984-d3bbd562921d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:09.039865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d029d14a-b446-4a0e-b984-d3bbd562921d admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:09.057668 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 907/1811] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:09.061398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca278e3f-5aa6-415e-975f-337179d14230 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:09.072970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca278e3f-5aa6-415e-975f-337179d14230 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:09.078760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77901cfa-4278-4d20-81af-22711ca55f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:09.079494 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77901cfa-4278-4d20-81af-22711ca55f8f admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:09.079779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77901cfa-4278-4d20-81af-22711ca55f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:09.080054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77901cfa-4278-4d20-81af-22711ca55f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:09.080944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77901cfa-4278-4d20-81af-22711ca55f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:09.082662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77901cfa-4278-4d20-81af-22711ca55f8f admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:09.105187 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029d14a-b446-4a0e-b984-d3bbd562921d admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:09.106141 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 905/1812] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:09.118416 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029d14a-b446-4a0e-b984-d3bbd562921d admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:09.137319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de2848-4d31-4594-b21e-f0c0fc9b1d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:09.137319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de2848-4d31-4594-b21e-f0c0fc9b1d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:09.137319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de2848-4d31-4594-b21e-f0c0fc9b1d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:09.137319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de2848-4d31-4594-b21e-f0c0fc9b1d9d admin admin] PATH_INFO: `/v3/projects/9f6945d7052f4833987fa295ec0c06fd/users/56ea498768154019a9a0c829e8673065/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:09.172061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de2848-4d31-4594-b21e-f0c0fc9b1d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f6945d7052f4833987fa295ec0c06fd, user_id=56ea498768154019a9a0c829e8673065, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:09.176588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de2848-4d31-4594-b21e-f0c0fc9b1d9d admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:09.202919 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 908/1813] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:09 2022] PUT /identity/v3/projects/9f6945d7052f4833987fa295ec0c06fd/users/56ea498768154019a9a0c829e8673065/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:09.215363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb30891-596a-49f7-a57c-b41343f68c44 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:09.215741 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb30891-596a-49f7-a57c-b41343f68c44 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:09.215977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb30891-596a-49f7-a57c-b41343f68c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:09.266759 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-4fb30891-596a-49f7-a57c-b41343f68c44 None None] MFA Rules not processed for user `56ea498768154019a9a0c829e8673065`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:09.268135 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4fb30891-596a-49f7-a57c-b41343f68c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:09.397812 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de2848-4d31-4594-b21e-f0c0fc9b1d9d admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:09.399689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a7de2848-4d31-4594-b21e-f0c0fc9b1d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:09.440127 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 906/1814] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:09.440127 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb30891-596a-49f7-a57c-b41343f68c44 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:09.569734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de2848-4d31-4594-b21e-f0c0fc9b1d9d admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:09.573712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d14114-8c40-4bc4-ba3d-1a7d745a0949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:09.574626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d14114-8c40-4bc4-ba3d-1a7d745a0949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:09.574899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d14114-8c40-4bc4-ba3d-1a7d745a0949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:09.575204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d14114-8c40-4bc4-ba3d-1a7d745a0949 admin admin] PATH_INFO: `/v3/users/c909ca17e90a4196bc21636a9f101b99` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:09.576501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d14114-8c40-4bc4-ba3d-1a7d745a0949 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c909ca17e90a4196bc21636a9f101b99)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:09.578301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb30891-596a-49f7-a57c-b41343f68c44 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:09.578436 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d14114-8c40-4bc4-ba3d-1a7d745a0949 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:09.584061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590539a6-cda1-4612-a687-a8c621643f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:09.585122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590539a6-cda1-4612-a687-a8c621643f00 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:09.585651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590539a6-cda1-4612-a687-a8c621643f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:09.586086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590539a6-cda1-4612-a687-a8c621643f00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:09.586797 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590539a6-cda1-4612-a687-a8c621643f00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:09.588943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590539a6-cda1-4612-a687-a8c621643f00 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:09.647941 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 907/1815] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:09 2022] POST /identity/v3/projects => generated 383 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:09.661486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590539a6-cda1-4612-a687-a8c621643f00 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:09.668698 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-78d14114-8c40-4bc4-ba3d-1a7d745a0949 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c909ca17e90a4196bc21636a9f101b99'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:09.691237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-78d14114-8c40-4bc4-ba3d-1a7d745a0949 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c909ca17e90a4196bc21636a9f101b99'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:09.700649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-78d14114-8c40-4bc4-ba3d-1a7d745a0949 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c909ca17e90a4196bc21636a9f101b99'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:09.701577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-78d14114-8c40-4bc4-ba3d-1a7d745a0949 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c909ca17e90a4196bc21636a9f101b99'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:09.702415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-78d14114-8c40-4bc4-ba3d-1a7d745a0949 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c909ca17e90a4196bc21636a9f101b99'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:09.743277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 909/1816] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:09 2022] DELETE /identity/v3/users/c909ca17e90a4196bc21636a9f101b99 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:09.748796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c0190-f023-4997-b842-5129dcc3ce0d None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:09.749090 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c0190-f023-4997-b842-5129dcc3ce0d None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:09.749386 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c0190-f023-4997-b842-5129dcc3ce0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:09.767978 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:09.786399 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:09.794428 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-435c0190-f023-4997-b842-5129dcc3ce0d None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:09.795219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590539a6-cda1-4612-a687-a8c621643f00 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:09.796089 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-435c0190-f023-4997-b842-5129dcc3ce0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:09.895984 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 910/1817] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:09.900827 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0e656-206d-4ec3-9eb4-65b24441e972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:09.901451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0e656-206d-4ec3-9eb4-65b24441e972 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:09.901451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0e656-206d-4ec3-9eb4-65b24441e972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:09.901748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0e656-206d-4ec3-9eb4-65b24441e972 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:09.902505 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0e656-206d-4ec3-9eb4-65b24441e972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:09.904748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0e656-206d-4ec3-9eb4-65b24441e972 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:09.911609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c0190-f023-4997-b842-5129dcc3ce0d None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:09.913324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-435c0190-f023-4997-b842-5129dcc3ce0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:09.946393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 908/1818] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:09 2022] POST /identity/v3/users => generated 298 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:09.948039 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0e656-206d-4ec3-9eb4-65b24441e972 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:09.949917 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-68b0e656-206d-4ec3-9eb4-65b24441e972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:09.968174 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0e656-206d-4ec3-9eb4-65b24441e972 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:09.973250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1486c-ae24-45a1-b095-f2994c1e1e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:09.974831 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1486c-ae24-45a1-b095-f2994c1e1e54 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:09.975146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1486c-ae24-45a1-b095-f2994c1e1e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:09.975917 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1486c-ae24-45a1-b095-f2994c1e1e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:09.976970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1486c-ae24-45a1-b095-f2994c1e1e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:09.979853 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1486c-ae24-45a1-b095-f2994c1e1e54 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:10.000391 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 909/1819] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:09 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:10.002194 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1486c-ae24-45a1-b095-f2994c1e1e54 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:10.022624 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c0190-f023-4997-b842-5129dcc3ce0d None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:10.023667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1486c-ae24-45a1-b095-f2994c1e1e54 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:10.031796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc819317-3dd8-4d6d-95cc-c7eda14b02d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:10.032673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc819317-3dd8-4d6d-95cc-c7eda14b02d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:10.032962 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc819317-3dd8-4d6d-95cc-c7eda14b02d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:10.033471 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc819317-3dd8-4d6d-95cc-c7eda14b02d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:10.037305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a5876-62d3-424a-99e3-4349d49d7e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:10.038457 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a5876-62d3-424a-99e3-4349d49d7e9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:10.038773 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a5876-62d3-424a-99e3-4349d49d7e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:10.040102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a5876-62d3-424a-99e3-4349d49d7e9b admin admin] PATH_INFO: `/v3/projects/31afd2ad2f604a3c9f74cfd68f4c9172/users/8bc2fb0f5f75423987af604635563d8b/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:10.059990 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc819317-3dd8-4d6d-95cc-c7eda14b02d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:10.061539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc819317-3dd8-4d6d-95cc-c7eda14b02d5 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:10.077658 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a5876-62d3-424a-99e3-4349d49d7e9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=31afd2ad2f604a3c9f74cfd68f4c9172, user_id=8bc2fb0f5f75423987af604635563d8b, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:10.083650 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a5876-62d3-424a-99e3-4349d49d7e9b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:10.089116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 911/1820] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:10.130661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 910/1821] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:10 2022] PUT /identity/v3/projects/31afd2ad2f604a3c9f74cfd68f4c9172/users/8bc2fb0f5f75423987af604635563d8b/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:10.135197 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc819317-3dd8-4d6d-95cc-c7eda14b02d5 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:10.156938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc819317-3dd8-4d6d-95cc-c7eda14b02d5 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:10.265415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc19c6f9-4ed7-41bf-a785-1efea33617b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:10.266212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19c6f9-4ed7-41bf-a785-1efea33617b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:10.266634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19c6f9-4ed7-41bf-a785-1efea33617b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:10.267050 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc19c6f9-4ed7-41bf-a785-1efea33617b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:10.268428 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc19c6f9-4ed7-41bf-a785-1efea33617b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:10.269893 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc19c6f9-4ed7-41bf-a785-1efea33617b9 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:10.286358 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 912/1822] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:10.288828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a5876-62d3-424a-99e3-4349d49d7e9b admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:10.308385 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a5876-62d3-424a-99e3-4349d49d7e9b admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:10.314249 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53561c39-3862-4f32-94b3-ecc59d9fad3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:10.315482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53561c39-3862-4f32-94b3-ecc59d9fad3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:10.315767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53561c39-3862-4f32-94b3-ecc59d9fad3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:10.316044 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53561c39-3862-4f32-94b3-ecc59d9fad3c admin admin] PATH_INFO: `/v3/projects/31afd2ad2f604a3c9f74cfd68f4c9172/users/8bc2fb0f5f75423987af604635563d8b/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:10.320291 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53561c39-3862-4f32-94b3-ecc59d9fad3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=31afd2ad2f604a3c9f74cfd68f4c9172, user_id=8bc2fb0f5f75423987af604635563d8b, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:10.325209 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53561c39-3862-4f32-94b3-ecc59d9fad3c admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:10.339731 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 911/1823] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:10 2022] PUT /identity/v3/projects/31afd2ad2f604a3c9f74cfd68f4c9172/users/8bc2fb0f5f75423987af604635563d8b/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:10.350005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866c3af-1d90-4473-a08f-10805de7b3a8 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:10.350639 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866c3af-1d90-4473-a08f-10805de7b3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:10.350973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866c3af-1d90-4473-a08f-10805de7b3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:10.409862 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-c866c3af-1d90-4473-a08f-10805de7b3a8 None None] MFA Rules not processed for user `8bc2fb0f5f75423987af604635563d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:10.417143 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-c866c3af-1d90-4473-a08f-10805de7b3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:10.506849 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:10.585935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 913/1824] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:10.609324 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53561c39-3862-4f32-94b3-ecc59d9fad3c admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:10.611586 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-53561c39-3862-4f32-94b3-ecc59d9fad3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:10.740379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53561c39-3862-4f32-94b3-ecc59d9fad3c admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:10.744871 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2759be-3351-4cad-8750-2ce5b407c432 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:10.745846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2759be-3351-4cad-8750-2ce5b407c432 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:10.746239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2759be-3351-4cad-8750-2ce5b407c432 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:10.746752 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2759be-3351-4cad-8750-2ce5b407c432 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:10.759438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2759be-3351-4cad-8750-2ce5b407c432 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:10.762025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2759be-3351-4cad-8750-2ce5b407c432 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:10.779442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 912/1825] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:11.392027 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866c3af-1d90-4473-a08f-10805de7b3a8 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:11.393857 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-c866c3af-1d90-4473-a08f-10805de7b3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:11.592792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866c3af-1d90-4473-a08f-10805de7b3a8 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:11.603119 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d099af-25fb-48bf-bb95-48f3be621e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:11.604032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d099af-25fb-48bf-bb95-48f3be621e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:11.604314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d099af-25fb-48bf-bb95-48f3be621e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:11.604610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d099af-25fb-48bf-bb95-48f3be621e24 admin admin] PATH_INFO: `/v3/projects/294cef7518074b2f8554bd35829af503` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:11.609012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d099af-25fb-48bf-bb95-48f3be621e24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=294cef7518074b2f8554bd35829af503)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:11.611025 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d099af-25fb-48bf-bb95-48f3be621e24 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:11.778535 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-54d099af-25fb-48bf-bb95-48f3be621e24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '294cef7518074b2f8554bd35829af503', 'request_id': 'req-54d099af-25fb-48bf-bb95-48f3be621e24'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:11.801606 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-54d099af-25fb-48bf-bb95-48f3be621e24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '294cef7518074b2f8554bd35829af503', 'request_id': 'req-54d099af-25fb-48bf-bb95-48f3be621e24'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:11.827924 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-54d099af-25fb-48bf-bb95-48f3be621e24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '294cef7518074b2f8554bd35829af503', 'request_id': 'req-54d099af-25fb-48bf-bb95-48f3be621e24'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:11.862334 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-54d099af-25fb-48bf-bb95-48f3be621e24 admin admin] The token cache is being invalidate because project 294cef7518074b2f8554bd35829af503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:11.862611 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-54d099af-25fb-48bf-bb95-48f3be621e24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:11.865148 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-54d099af-25fb-48bf-bb95-48f3be621e24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:11.870380 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 914/1826] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:11 2022] DELETE /identity/v3/projects/294cef7518074b2f8554bd35829af503 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:11.889057 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2759be-3351-4cad-8750-2ce5b407c432 service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:11.904841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2759be-3351-4cad-8750-2ce5b407c432 service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:12.043354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:12.127260 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445e9d2-d7cf-4bd6-a493-52a3c16176a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:12.127917 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445e9d2-d7cf-4bd6-a493-52a3c16176a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:12.128206 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445e9d2-d7cf-4bd6-a493-52a3c16176a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:12.128484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445e9d2-d7cf-4bd6-a493-52a3c16176a2 admin admin] PATH_INFO: `/v3/users/3e8509e8358c462f89783dea99e00acc` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:12.133499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445e9d2-d7cf-4bd6-a493-52a3c16176a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e8509e8358c462f89783dea99e00acc)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:12.141032 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445e9d2-d7cf-4bd6-a493-52a3c16176a2 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:12.261994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c445e9d2-d7cf-4bd6-a493-52a3c16176a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e8509e8358c462f89783dea99e00acc'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:12.294727 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c445e9d2-d7cf-4bd6-a493-52a3c16176a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e8509e8358c462f89783dea99e00acc'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:12.308050 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d099af-25fb-48bf-bb95-48f3be621e24 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:12.310042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-54d099af-25fb-48bf-bb95-48f3be621e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:12.315153 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c445e9d2-d7cf-4bd6-a493-52a3c16176a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e8509e8358c462f89783dea99e00acc'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:12.315883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c445e9d2-d7cf-4bd6-a493-52a3c16176a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e8509e8358c462f89783dea99e00acc'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:12.316499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c445e9d2-d7cf-4bd6-a493-52a3c16176a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e8509e8358c462f89783dea99e00acc'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:12.343488 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 913/1827] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:11 2022] DELETE /identity/v3/users/3e8509e8358c462f89783dea99e00acc => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:12.442454 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d099af-25fb-48bf-bb95-48f3be621e24 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:12.456937 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:12.538173 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4706931-3629-4d63-94aa-7bfb36862fd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:12.538934 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4706931-3629-4d63-94aa-7bfb36862fd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:12.539303 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4706931-3629-4d63-94aa-7bfb36862fd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:12.540213 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4706931-3629-4d63-94aa-7bfb36862fd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:12.544577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-c4706931-3629-4d63-94aa-7bfb36862fd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:12.675098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4706931-3629-4d63-94aa-7bfb36862fd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:12.679137 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4706931-3629-4d63-94aa-7bfb36862fd5 service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:12.696216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 915/1828] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:12.709174 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71bc01-14f9-4dff-98a6-2d6954585b51 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:12.709488 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71bc01-14f9-4dff-98a6-2d6954585b51 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:12.709750 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71bc01-14f9-4dff-98a6-2d6954585b51 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:12.715447 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 914/1829] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jul 21 05:44:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:44:12.753730 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91c8bc-f439-41a4-bdb4-25b8b8ab4ea9 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:12.754025 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91c8bc-f439-41a4-bdb4-25b8b8ab4ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:12.754375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91c8bc-f439-41a4-bdb4-25b8b8ab4ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:12.798418 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-5e91c8bc-f439-41a4-bdb4-25b8b8ab4ea9 None None] MFA Rules not processed for user `928d014648cd41d7b58e7e01845eb748`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:12.800654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5e91c8bc-f439-41a4-bdb4-25b8b8ab4ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:12.958749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 916/1830] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jul 21 05:44:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:44:12.981848 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71bc01-14f9-4dff-98a6-2d6954585b51 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:12.984443 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-cf71bc01-14f9-4dff-98a6-2d6954585b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:13.029713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71bc01-14f9-4dff-98a6-2d6954585b51 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:13.042001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2709724-bf2f-4335-a847-bc592e8ebc10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:13.042674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2709724-bf2f-4335-a847-bc592e8ebc10 service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:13.042965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2709724-bf2f-4335-a847-bc592e8ebc10 service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:13.043338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2709724-bf2f-4335-a847-bc592e8ebc10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:13.067151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2709724-bf2f-4335-a847-bc592e8ebc10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:13.068716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2709724-bf2f-4335-a847-bc592e8ebc10 service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:13.083552 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 915/1831] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:13.101160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91c8bc-f439-41a4-bdb4-25b8b8ab4ea9 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:13.102768 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5e91c8bc-f439-41a4-bdb4-25b8b8ab4ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:13.266466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91c8bc-f439-41a4-bdb4-25b8b8ab4ea9 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:13.276711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa18bd3-2fea-49dc-ae61-942b19f4851a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:13.277964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa18bd3-2fea-49dc-ae61-942b19f4851a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:13.278659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa18bd3-2fea-49dc-ae61-942b19f4851a admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:13.282630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa18bd3-2fea-49dc-ae61-942b19f4851a admin admin] PATH_INFO: `/v3/projects/894f846ef7394eeebc695b698669cd56` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:13.284211 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa18bd3-2fea-49dc-ae61-942b19f4851a admin admin] RBAC: Authorizing `identity:delete_project(project_id=894f846ef7394eeebc695b698669cd56)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:13.295258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa18bd3-2fea-49dc-ae61-942b19f4851a admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:13.308812 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:13.412539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-7fa18bd3-2fea-49dc-ae61-942b19f4851a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '894f846ef7394eeebc695b698669cd56', 'request_id': 'req-7fa18bd3-2fea-49dc-ae61-942b19f4851a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:13.416645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-7fa18bd3-2fea-49dc-ae61-942b19f4851a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '894f846ef7394eeebc695b698669cd56', 'request_id': 'req-7fa18bd3-2fea-49dc-ae61-942b19f4851a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:13.424009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-7fa18bd3-2fea-49dc-ae61-942b19f4851a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '894f846ef7394eeebc695b698669cd56', 'request_id': 'req-7fa18bd3-2fea-49dc-ae61-942b19f4851a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:13.450848 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-7fa18bd3-2fea-49dc-ae61-942b19f4851a admin admin] The token cache is being invalidate because project 894f846ef7394eeebc695b698669cd56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:13.451870 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-7fa18bd3-2fea-49dc-ae61-942b19f4851a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:13.453855 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-7fa18bd3-2fea-49dc-ae61-942b19f4851a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:13.463515 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 917/1832] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:13 2022] DELETE /identity/v3/projects/894f846ef7394eeebc695b698669cd56 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:13.465910 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260c308-2838-4868-b15b-b927ab837d4b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:13.466343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260c308-2838-4868-b15b-b927ab837d4b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:13.466627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260c308-2838-4868-b15b-b927ab837d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:13.489466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-b260c308-2838-4868-b15b-b927ab837d4b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:13.490702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b260c308-2838-4868-b15b-b927ab837d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:13.616396 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 916/1833] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:13.627384 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30cf16-500b-4cce-a4c5-c4ed929ce5d3 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:13.627746 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30cf16-500b-4cce-a4c5-c4ed929ce5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:13.628068 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30cf16-500b-4cce-a4c5-c4ed929ce5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:13.662409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-4b30cf16-500b-4cce-a4c5-c4ed929ce5d3 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:13.664032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4b30cf16-500b-4cce-a4c5-c4ed929ce5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:13.683259 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 918/1834] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:13.684870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260c308-2838-4868-b15b-b927ab837d4b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:13.699165 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260c308-2838-4868-b15b-b927ab837d4b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:13.714116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a6a68-21a6-4cb4-8733-b19f8914a2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:13.714806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a6a68-21a6-4cb4-8733-b19f8914a2f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:13.715128 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a6a68-21a6-4cb4-8733-b19f8914a2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:13.715407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a6a68-21a6-4cb4-8733-b19f8914a2f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:13.720500 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a6a68-21a6-4cb4-8733-b19f8914a2f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:44:13.721205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a6a68-21a6-4cb4-8733-b19f8914a2f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:13.722445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a6a68-21a6-4cb4-8733-b19f8914a2f8 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:13.755853 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 917/1835] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:44:13.760627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30cf16-500b-4cce-a4c5-c4ed929ce5d3 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:13.771555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30cf16-500b-4cce-a4c5-c4ed929ce5d3 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:13.777260 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde7a56-39d9-4053-8250-c16949e4e74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:13.777957 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde7a56-39d9-4053-8250-c16949e4e74c admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:13.778236 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde7a56-39d9-4053-8250-c16949e4e74c admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:13.778507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde7a56-39d9-4053-8250-c16949e4e74c admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:13.779289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde7a56-39d9-4053-8250-c16949e4e74c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:13.781304 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde7a56-39d9-4053-8250-c16949e4e74c admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:13.843121 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 919/1836] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:13 2022] POST /identity/v3/projects => generated 391 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:44:13.849910 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a6a68-21a6-4cb4-8733-b19f8914a2f8 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:13.867930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a6a68-21a6-4cb4-8733-b19f8914a2f8 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:13.909635 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:13.941736 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83015c1a-a25c-4613-a612-503afe6d9542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:13.942338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83015c1a-a25c-4613-a612-503afe6d9542 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:13.942618 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83015c1a-a25c-4613-a612-503afe6d9542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:13.942890 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83015c1a-a25c-4613-a612-503afe6d9542 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:13.943621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83015c1a-a25c-4613-a612-503afe6d9542 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:13.945620 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83015c1a-a25c-4613-a612-503afe6d9542 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:13.966944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 918/1837] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:13 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:13.971588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde7a56-39d9-4053-8250-c16949e4e74c admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:13.991064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde7a56-39d9-4053-8250-c16949e4e74c admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:13.991064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf3a60-e996-40c5-936f-b635511e006b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:13.991064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf3a60-e996-40c5-936f-b635511e006b admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:13.991064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf3a60-e996-40c5-936f-b635511e006b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:13.991511 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf3a60-e996-40c5-936f-b635511e006b admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:13.991511 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf3a60-e996-40c5-936f-b635511e006b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:13.992305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf3a60-e996-40c5-936f-b635511e006b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:14.004609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 920/1838] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:14.009740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83015c1a-a25c-4613-a612-503afe6d9542 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:14.020968 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83015c1a-a25c-4613-a612-503afe6d9542 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:14.026365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124fe99-3856-4263-8569-d4663888827f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:14.027093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124fe99-3856-4263-8569-d4663888827f admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:14.027423 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124fe99-3856-4263-8569-d4663888827f admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:14.027804 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124fe99-3856-4263-8569-d4663888827f admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:14.028549 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124fe99-3856-4263-8569-d4663888827f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:14.030274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124fe99-3856-4263-8569-d4663888827f admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:14.043784 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 919/1839] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:14.048009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf3a60-e996-40c5-936f-b635511e006b admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:14.063586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf3a60-e996-40c5-936f-b635511e006b admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:14.068448 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f02eec-94ef-4964-8a58-ae6dd78c4b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:14.069355 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f02eec-94ef-4964-8a58-ae6dd78c4b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:14.069643 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f02eec-94ef-4964-8a58-ae6dd78c4b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:14.069916 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f02eec-94ef-4964-8a58-ae6dd78c4b37 admin admin] PATH_INFO: `/v3/projects/b2b42f77e8cd41f086eb703be44f849c/users/0e764bab188a44649405650d8d7df3a4/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:14.084620 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f02eec-94ef-4964-8a58-ae6dd78c4b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b42f77e8cd41f086eb703be44f849c, user_id=0e764bab188a44649405650d8d7df3a4, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:14.087526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f02eec-94ef-4964-8a58-ae6dd78c4b37 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:14.102252 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 921/1840] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:14 2022] PUT /identity/v3/projects/b2b42f77e8cd41f086eb703be44f849c/users/0e764bab188a44649405650d8d7df3a4/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:14.112905 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc63c9a-1250-473c-bb1b-00b79115af2c None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:14.113196 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc63c9a-1250-473c-bb1b-00b79115af2c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:14.113480 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc63c9a-1250-473c-bb1b-00b79115af2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:14.140325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-9bc63c9a-1250-473c-bb1b-00b79115af2c None None] MFA Rules not processed for user `0e764bab188a44649405650d8d7df3a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:14.141531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-9bc63c9a-1250-473c-bb1b-00b79115af2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:14.207606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 920/1841] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:14.214533 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f02eec-94ef-4964-8a58-ae6dd78c4b37 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:14.280726 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f02eec-94ef-4964-8a58-ae6dd78c4b37 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:14.286754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406d158-d2fd-41ab-8c9c-fd02ddb27486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:14.287455 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406d158-d2fd-41ab-8c9c-fd02ddb27486 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:14.287739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406d158-d2fd-41ab-8c9c-fd02ddb27486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:14.288021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406d158-d2fd-41ab-8c9c-fd02ddb27486 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:14.288704 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406d158-d2fd-41ab-8c9c-fd02ddb27486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:14.290636 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406d158-d2fd-41ab-8c9c-fd02ddb27486 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:14.321698 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 922/1842] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:14 2022] POST /identity/v3/projects => generated 391 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:14.326029 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc63c9a-1250-473c-bb1b-00b79115af2c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:14.349899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc63c9a-1250-473c-bb1b-00b79115af2c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:14.414013 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb59a2-8ba9-4087-8d04-4b23fab68cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:14.414828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb59a2-8ba9-4087-8d04-4b23fab68cab admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:14.415374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb59a2-8ba9-4087-8d04-4b23fab68cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:14.415570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb59a2-8ba9-4087-8d04-4b23fab68cab admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:14.416316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb59a2-8ba9-4087-8d04-4b23fab68cab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:14.418393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb59a2-8ba9-4087-8d04-4b23fab68cab admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:14.441038 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 921/1843] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:14 2022] POST /identity/v3/users => generated 302 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:14.445270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406d158-d2fd-41ab-8c9c-fd02ddb27486 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:14.456290 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406d158-d2fd-41ab-8c9c-fd02ddb27486 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:14.462406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc37447a-6d82-4dbc-812a-681af7c6aa47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:14.463125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37447a-6d82-4dbc-812a-681af7c6aa47 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:14.463414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37447a-6d82-4dbc-812a-681af7c6aa47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:14.463696 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37447a-6d82-4dbc-812a-681af7c6aa47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:14.464470 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc37447a-6d82-4dbc-812a-681af7c6aa47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:14.466062 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc37447a-6d82-4dbc-812a-681af7c6aa47 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:14.477867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 923/1844] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:14.481948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb59a2-8ba9-4087-8d04-4b23fab68cab admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:14.492111 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb59a2-8ba9-4087-8d04-4b23fab68cab admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:14.496799 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084a3a8-4816-4fdb-a98c-b23e8c0eb856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:14.497678 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084a3a8-4816-4fdb-a98c-b23e8c0eb856 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:14.497981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084a3a8-4816-4fdb-a98c-b23e8c0eb856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:14.498262 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084a3a8-4816-4fdb-a98c-b23e8c0eb856 admin admin] PATH_INFO: `/v3/projects/ae5cfd2d297b45728ba0b7a99ac71278/users/8157a16f4059486e805fce80b582ebbb/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:14.514265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084a3a8-4816-4fdb-a98c-b23e8c0eb856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae5cfd2d297b45728ba0b7a99ac71278, user_id=8157a16f4059486e805fce80b582ebbb, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:14.517130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084a3a8-4816-4fdb-a98c-b23e8c0eb856 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:14.532690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 922/1845] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:14 2022] PUT /identity/v3/projects/ae5cfd2d297b45728ba0b7a99ac71278/users/8157a16f4059486e805fce80b582ebbb/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:14.537176 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc37447a-6d82-4dbc-812a-681af7c6aa47 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:14.543678 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:14.549200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc37447a-6d82-4dbc-812a-681af7c6aa47 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:14.611933 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e55571-a361-4ac4-829a-ce8576b9b614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:14.612764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e55571-a361-4ac4-829a-ce8576b9b614 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:14.613637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e55571-a361-4ac4-829a-ce8576b9b614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:14.613930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e55571-a361-4ac4-829a-ce8576b9b614 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:14.614758 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e55571-a361-4ac4-829a-ce8576b9b614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:14.616924 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e55571-a361-4ac4-829a-ce8576b9b614 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:14.629297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 924/1846] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:14.633876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084a3a8-4816-4fdb-a98c-b23e8c0eb856 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:14.642661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:14.651215 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084a3a8-4816-4fdb-a98c-b23e8c0eb856 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:14.672652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c1748-f15e-4cce-9269-5b40c6cb1e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:14.673677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c1748-f15e-4cce-9269-5b40c6cb1e4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:14.674032 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c1748-f15e-4cce-9269-5b40c6cb1e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:14.674381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c1748-f15e-4cce-9269-5b40c6cb1e4f admin admin] PATH_INFO: `/v3/projects/ae5cfd2d297b45728ba0b7a99ac71278/users/8157a16f4059486e805fce80b582ebbb/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:14.694150 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c1748-f15e-4cce-9269-5b40c6cb1e4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae5cfd2d297b45728ba0b7a99ac71278, user_id=8157a16f4059486e805fce80b582ebbb, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:14.697296 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c1748-f15e-4cce-9269-5b40c6cb1e4f admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:14.721451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 923/1847] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:14 2022] PUT /identity/v3/projects/ae5cfd2d297b45728ba0b7a99ac71278/users/8157a16f4059486e805fce80b582ebbb/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:14.732440 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6bb53-6810-4347-ad67-b65599fd0095 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:14.732860 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6bb53-6810-4347-ad67-b65599fd0095 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:14.733167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6bb53-6810-4347-ad67-b65599fd0095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:14.793871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-b8d6bb53-6810-4347-ad67-b65599fd0095 None None] MFA Rules not processed for user `8157a16f4059486e805fce80b582ebbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:14.795665 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b8d6bb53-6810-4347-ad67-b65599fd0095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:14.909600 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 925/1848] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:14.923690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c1748-f15e-4cce-9269-5b40c6cb1e4f admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:14.925485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b54c1748-f15e-4cce-9269-5b40c6cb1e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:15.019670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c1748-f15e-4cce-9269-5b40c6cb1e4f admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:15.023399 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7f0c6-3d82-4eee-8889-80bceee9760a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:15.024060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7f0c6-3d82-4eee-8889-80bceee9760a service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:15.024341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7f0c6-3d82-4eee-8889-80bceee9760a service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:15.024613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7f0c6-3d82-4eee-8889-80bceee9760a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:15.034481 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7f0c6-3d82-4eee-8889-80bceee9760a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:15.036350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac7f0c6-3d82-4eee-8889-80bceee9760a service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:15.049194 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 924/1849] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:15.112498 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6bb53-6810-4347-ad67-b65599fd0095 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:15.114027 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b8d6bb53-6810-4347-ad67-b65599fd0095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:15.131717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6bb53-6810-4347-ad67-b65599fd0095 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:15.135805 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d1d9d-4a69-4cba-b477-d236c842b37c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:15.136479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d1d9d-4a69-4cba-b477-d236c842b37c service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:15.136768 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d1d9d-4a69-4cba-b477-d236c842b37c service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:15.137043 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d1d9d-4a69-4cba-b477-d236c842b37c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:15.204039 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d1d9d-4a69-4cba-b477-d236c842b37c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:15.205599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d1d9d-4a69-4cba-b477-d236c842b37c service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:15.223160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 926/1850] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:18.733177 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7f0c6-3d82-4eee-8889-80bceee9760a service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:18.737769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-7ac7f0c6-3d82-4eee-8889-80bceee9760a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:18.811878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7f0c6-3d82-4eee-8889-80bceee9760a service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:18.819748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4055c-f44f-472f-8eb3-0bb9325b5dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:18.820700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4055c-f44f-472f-8eb3-0bb9325b5dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:18.820983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4055c-f44f-472f-8eb3-0bb9325b5dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:18.821265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4055c-f44f-472f-8eb3-0bb9325b5dfe admin admin] PATH_INFO: `/v3/users/a031645424364c9c9658724093f0caf5` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:18.822513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4055c-f44f-472f-8eb3-0bb9325b5dfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=a031645424364c9c9658724093f0caf5)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:18.824582 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4055c-f44f-472f-8eb3-0bb9325b5dfe admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:18.884093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-46e4055c-f44f-472f-8eb3-0bb9325b5dfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a031645424364c9c9658724093f0caf5'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:18.900751 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-46e4055c-f44f-472f-8eb3-0bb9325b5dfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a031645424364c9c9658724093f0caf5'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:18.910918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-46e4055c-f44f-472f-8eb3-0bb9325b5dfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a031645424364c9c9658724093f0caf5'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:18.912149 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-46e4055c-f44f-472f-8eb3-0bb9325b5dfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a031645424364c9c9658724093f0caf5'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:18.912816 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-46e4055c-f44f-472f-8eb3-0bb9325b5dfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a031645424364c9c9658724093f0caf5'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:18.929298 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 925/1851] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:18 2022] DELETE /identity/v3/users/a031645424364c9c9658724093f0caf5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:18.934467 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722b4b4-832e-4230-b3be-cefa925d8776 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:18.935516 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722b4b4-832e-4230-b3be-cefa925d8776 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:18.936581 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722b4b4-832e-4230-b3be-cefa925d8776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:18.957604 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-3722b4b4-832e-4230-b3be-cefa925d8776 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:18.959139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3722b4b4-832e-4230-b3be-cefa925d8776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:18.983178 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:19.028191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 927/1852] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:44:19.039451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4055c-f44f-472f-8eb3-0bb9325b5dfe admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:19.041018 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-46e4055c-f44f-472f-8eb3-0bb9325b5dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:19.056009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:19.101536 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4055c-f44f-472f-8eb3-0bb9325b5dfe admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:19.109643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027bfa1-03e2-4dfb-a681-d12c7be591f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:19.109643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027bfa1-03e2-4dfb-a681-d12c7be591f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:19.109643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027bfa1-03e2-4dfb-a681-d12c7be591f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:19.109643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027bfa1-03e2-4dfb-a681-d12c7be591f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:19.120101 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027bfa1-03e2-4dfb-a681-d12c7be591f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:19.121564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027bfa1-03e2-4dfb-a681-d12c7be591f5 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:19.131866 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 926/1853] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:19.423547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722b4b4-832e-4230-b3be-cefa925d8776 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:19.425284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3722b4b4-832e-4230-b3be-cefa925d8776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:19.440706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722b4b4-832e-4230-b3be-cefa925d8776 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:19.446831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ec215-96a2-455b-b966-b739492e615f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:19.447845 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ec215-96a2-455b-b966-b739492e615f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:19.448198 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ec215-96a2-455b-b966-b739492e615f admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:19.448544 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ec215-96a2-455b-b966-b739492e615f admin admin] PATH_INFO: `/v3/projects/733bf3e2212e47c1a7051b532f812e61` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:19.450035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ec215-96a2-455b-b966-b739492e615f admin admin] RBAC: Authorizing `identity:delete_project(project_id=733bf3e2212e47c1a7051b532f812e61)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:19.452170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ec215-96a2-455b-b966-b739492e615f admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:19.521397 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-399ec215-96a2-455b-b966-b739492e615f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '733bf3e2212e47c1a7051b532f812e61', 'request_id': 'req-399ec215-96a2-455b-b966-b739492e615f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:19.530991 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-399ec215-96a2-455b-b966-b739492e615f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '733bf3e2212e47c1a7051b532f812e61', 'request_id': 'req-399ec215-96a2-455b-b966-b739492e615f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:19.538731 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-399ec215-96a2-455b-b966-b739492e615f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '733bf3e2212e47c1a7051b532f812e61', 'request_id': 'req-399ec215-96a2-455b-b966-b739492e615f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:19.556312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-399ec215-96a2-455b-b966-b739492e615f admin admin] The token cache is being invalidate because project 733bf3e2212e47c1a7051b532f812e61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:19.556624 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-399ec215-96a2-455b-b966-b739492e615f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:19.557446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-399ec215-96a2-455b-b966-b739492e615f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:19.559187 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 928/1854] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:19 2022] DELETE /identity/v3/projects/733bf3e2212e47c1a7051b532f812e61 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:19.563915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027bfa1-03e2-4dfb-a681-d12c7be591f5 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:19.574090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027bfa1-03e2-4dfb-a681-d12c7be591f5 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:19.655915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e025e-243e-45fc-b3b7-2d819157f436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:19.655915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e025e-243e-45fc-b3b7-2d819157f436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:19.656216 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e025e-243e-45fc-b3b7-2d819157f436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:19.659673 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e025e-243e-45fc-b3b7-2d819157f436 admin admin] PATH_INFO: `/v3/users/a1362159194a4b948d3ff67e085d2b8e` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:19.662252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e025e-243e-45fc-b3b7-2d819157f436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1362159194a4b948d3ff67e085d2b8e)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:19.666468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e025e-243e-45fc-b3b7-2d819157f436 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:19.768039 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-fe9e025e-243e-45fc-b3b7-2d819157f436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1362159194a4b948d3ff67e085d2b8e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:19.791630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-fe9e025e-243e-45fc-b3b7-2d819157f436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1362159194a4b948d3ff67e085d2b8e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:19.800457 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-fe9e025e-243e-45fc-b3b7-2d819157f436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1362159194a4b948d3ff67e085d2b8e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:19.802008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-fe9e025e-243e-45fc-b3b7-2d819157f436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1362159194a4b948d3ff67e085d2b8e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:19.803105 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-fe9e025e-243e-45fc-b3b7-2d819157f436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1362159194a4b948d3ff67e085d2b8e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:19.819950 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 927/1855] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:19 2022] DELETE /identity/v3/users/a1362159194a4b948d3ff67e085d2b8e => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:20.383788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ec215-96a2-455b-b966-b739492e615f admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:20.385397 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-399ec215-96a2-455b-b966-b739492e615f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:20.399029 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ec215-96a2-455b-b966-b739492e615f admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:20.458229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0513468-82b6-4f79-a759-7fb495421b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:20.459151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0513468-82b6-4f79-a759-7fb495421b7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:20.459436 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0513468-82b6-4f79-a759-7fb495421b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:20.459707 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0513468-82b6-4f79-a759-7fb495421b7a admin admin] PATH_INFO: `/v3/projects/64a03e578f904d00a1e7d4c48293e2ed` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:20.460916 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0513468-82b6-4f79-a759-7fb495421b7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=64a03e578f904d00a1e7d4c48293e2ed)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:20.462883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0513468-82b6-4f79-a759-7fb495421b7a admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:20.515970 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f0513468-82b6-4f79-a759-7fb495421b7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64a03e578f904d00a1e7d4c48293e2ed', 'request_id': 'req-f0513468-82b6-4f79-a759-7fb495421b7a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:20.517274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:20.525467 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f0513468-82b6-4f79-a759-7fb495421b7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64a03e578f904d00a1e7d4c48293e2ed', 'request_id': 'req-f0513468-82b6-4f79-a759-7fb495421b7a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:20.534408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f0513468-82b6-4f79-a759-7fb495421b7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64a03e578f904d00a1e7d4c48293e2ed', 'request_id': 'req-f0513468-82b6-4f79-a759-7fb495421b7a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:20.552865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f0513468-82b6-4f79-a759-7fb495421b7a admin admin] The token cache is being invalidate because project 64a03e578f904d00a1e7d4c48293e2ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:20.553177 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f0513468-82b6-4f79-a759-7fb495421b7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:20.553851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f0513468-82b6-4f79-a759-7fb495421b7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:20.555548 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 929/1856] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:20 2022] DELETE /identity/v3/projects/64a03e578f904d00a1e7d4c48293e2ed => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:20.561410 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff4ace-18fc-423d-9692-023c62f7f525 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:20.561716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff4ace-18fc-423d-9692-023c62f7f525 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:20.562118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff4ace-18fc-423d-9692-023c62f7f525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:20.585870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-1eff4ace-18fc-423d-9692-023c62f7f525 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:20.587145 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1eff4ace-18fc-423d-9692-023c62f7f525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:20.624074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:20.657887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 928/1857] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:20.666214 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114de0a5-da5c-4fdd-8b39-371a9fceb08f None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:20.666539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114de0a5-da5c-4fdd-8b39-371a9fceb08f None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:20.666823 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114de0a5-da5c-4fdd-8b39-371a9fceb08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:20.685627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-114de0a5-da5c-4fdd-8b39-371a9fceb08f None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:20.686820 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-114de0a5-da5c-4fdd-8b39-371a9fceb08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:20.694715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 930/1858] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:20.699034 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff4ace-18fc-423d-9692-023c62f7f525 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:20.709278 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff4ace-18fc-423d-9692-023c62f7f525 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:20.714291 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaef401-d955-4c40-8aba-77c2e4ba11c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:20.715178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaef401-d955-4c40-8aba-77c2e4ba11c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:20.715296 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaef401-d955-4c40-8aba-77c2e4ba11c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:20.715607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaef401-d955-4c40-8aba-77c2e4ba11c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:20.716471 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaef401-d955-4c40-8aba-77c2e4ba11c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:44:20.716786 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaef401-d955-4c40-8aba-77c2e4ba11c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:20.718417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaef401-d955-4c40-8aba-77c2e4ba11c3 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:20.731401 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 929/1859] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:44:20.738386 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114de0a5-da5c-4fdd-8b39-371a9fceb08f None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:20.748255 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114de0a5-da5c-4fdd-8b39-371a9fceb08f None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:20.756805 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c0314-2ff0-4a07-a400-76ed1ffd0772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:20.757695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0314-2ff0-4a07-a400-76ed1ffd0772 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:20.757982 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0314-2ff0-4a07-a400-76ed1ffd0772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:20.758258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0314-2ff0-4a07-a400-76ed1ffd0772 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:20.758970 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c0314-2ff0-4a07-a400-76ed1ffd0772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:20.760996 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c0314-2ff0-4a07-a400-76ed1ffd0772 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:20.800395 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 931/1860] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:20 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:20.804810 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaef401-d955-4c40-8aba-77c2e4ba11c3 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:20.816560 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaef401-d955-4c40-8aba-77c2e4ba11c3 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:20.894734 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac341d4-6e18-4f49-ba62-e0c87c5e8767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:20.895520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac341d4-6e18-4f49-ba62-e0c87c5e8767 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:20.895768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac341d4-6e18-4f49-ba62-e0c87c5e8767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:20.897984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac341d4-6e18-4f49-ba62-e0c87c5e8767 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:20.899889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac341d4-6e18-4f49-ba62-e0c87c5e8767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:20.902104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c0314-2ff0-4a07-a400-76ed1ffd0772 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:20.903253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac341d4-6e18-4f49-ba62-e0c87c5e8767 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:20.903966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-2a8c0314-2ff0-4a07-a400-76ed1ffd0772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:20.920040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c0314-2ff0-4a07-a400-76ed1ffd0772 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:20.933354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 930/1861] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:20 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:44:20.934638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac341d4-6e18-4f49-ba62-e0c87c5e8767 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:20.934901 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4541f-c43c-457a-83b2-997659bfb6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:20.936344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4541f-c43c-457a-83b2-997659bfb6cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:20.936799 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4541f-c43c-457a-83b2-997659bfb6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:20.937300 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4541f-c43c-457a-83b2-997659bfb6cc admin admin] PATH_INFO: `/v3/users/22b9864177804ce590571c8e8ee412dc` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:20.941505 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb4541f-c43c-457a-83b2-997659bfb6cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=22b9864177804ce590571c8e8ee412dc)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:20.945700 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb4541f-c43c-457a-83b2-997659bfb6cc admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:20.947420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac341d4-6e18-4f49-ba62-e0c87c5e8767 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:20.954686 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0177e-dbdd-478c-8882-80a3f1e3dec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:20.954686 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0177e-dbdd-478c-8882-80a3f1e3dec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:20.954686 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0177e-dbdd-478c-8882-80a3f1e3dec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:20.954686 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0177e-dbdd-478c-8882-80a3f1e3dec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:20.955322 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a0177e-dbdd-478c-8882-80a3f1e3dec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:20.956938 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a0177e-dbdd-478c-8882-80a3f1e3dec6 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:20.978469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 931/1862] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:20.983028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0177e-dbdd-478c-8882-80a3f1e3dec6 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:20.999398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0177e-dbdd-478c-8882-80a3f1e3dec6 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:21.010808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0398a-6a9f-423b-ac19-6f474a3d2544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:21.011915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0398a-6a9f-423b-ac19-6f474a3d2544 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:21.012486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0398a-6a9f-423b-ac19-6f474a3d2544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:21.012993 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0398a-6a9f-423b-ac19-6f474a3d2544 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:21.013975 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0398a-6a9f-423b-ac19-6f474a3d2544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:21.016184 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0398a-6a9f-423b-ac19-6f474a3d2544 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:21.031401 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 932/1863] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:21.032633 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0398a-6a9f-423b-ac19-6f474a3d2544 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:21.044849 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-13f0398a-6a9f-423b-ac19-6f474a3d2544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:21.050815 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-dbb4541f-c43c-457a-83b2-997659bfb6cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22b9864177804ce590571c8e8ee412dc'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:21.059193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0398a-6a9f-423b-ac19-6f474a3d2544 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:21.067648 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456da0e-da27-4b56-9160-32c62d131ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:21.068547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456da0e-da27-4b56-9160-32c62d131ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:21.068841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456da0e-da27-4b56-9160-32c62d131ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:21.069158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456da0e-da27-4b56-9160-32c62d131ce8 admin admin] PATH_INFO: `/v3/users/0e764bab188a44649405650d8d7df3a4` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:21.070523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456da0e-da27-4b56-9160-32c62d131ce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e764bab188a44649405650d8d7df3a4)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:21.071720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-dbb4541f-c43c-457a-83b2-997659bfb6cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22b9864177804ce590571c8e8ee412dc'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:21.076582 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456da0e-da27-4b56-9160-32c62d131ce8 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:21.085761 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-dbb4541f-c43c-457a-83b2-997659bfb6cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22b9864177804ce590571c8e8ee412dc'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:21.086964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-dbb4541f-c43c-457a-83b2-997659bfb6cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22b9864177804ce590571c8e8ee412dc'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:21.088965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-dbb4541f-c43c-457a-83b2-997659bfb6cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22b9864177804ce590571c8e8ee412dc'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:21.119605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 932/1864] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:20 2022] DELETE /identity/v3/users/22b9864177804ce590571c8e8ee412dc => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:21.121474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4541f-c43c-457a-83b2-997659bfb6cc admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:21.124061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-dbb4541f-c43c-457a-83b2-997659bfb6cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:21.163766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a456da0e-da27-4b56-9160-32c62d131ce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e764bab188a44649405650d8d7df3a4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:21.204396 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a456da0e-da27-4b56-9160-32c62d131ce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e764bab188a44649405650d8d7df3a4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:21.212301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a456da0e-da27-4b56-9160-32c62d131ce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e764bab188a44649405650d8d7df3a4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:21.213548 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a456da0e-da27-4b56-9160-32c62d131ce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e764bab188a44649405650d8d7df3a4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:21.215689 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a456da0e-da27-4b56-9160-32c62d131ce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e764bab188a44649405650d8d7df3a4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:21.225729 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4541f-c43c-457a-83b2-997659bfb6cc admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:21.233096 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f6e46-c574-4eef-8d4f-160972276234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:21.234669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f6e46-c574-4eef-8d4f-160972276234 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:21.235555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f6e46-c574-4eef-8d4f-160972276234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:21.236013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f6e46-c574-4eef-8d4f-160972276234 admin admin] PATH_INFO: `/v3/projects/c0ef8176a9224b14b444ee66057cca20/users/c3308ce8e3634d1693b34a912af386d1/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:21.242956 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 933/1865] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:21 2022] DELETE /identity/v3/users/0e764bab188a44649405650d8d7df3a4 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:21.244844 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa724548-9c38-4c21-ace9-cc162a214d63 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:21.245349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa724548-9c38-4c21-ace9-cc162a214d63 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:21.245847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa724548-9c38-4c21-ace9-cc162a214d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:21.253631 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:21.255917 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f6e46-c574-4eef-8d4f-160972276234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ef8176a9224b14b444ee66057cca20, user_id=c3308ce8e3634d1693b34a912af386d1, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:21.259817 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f6e46-c574-4eef-8d4f-160972276234 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:21.277558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-aa724548-9c38-4c21-ace9-cc162a214d63 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:21.279486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-aa724548-9c38-4c21-ace9-cc162a214d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:21.281222 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 933/1866] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:21 2022] PUT /identity/v3/projects/c0ef8176a9224b14b444ee66057cca20/users/c3308ce8e3634d1693b34a912af386d1/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:21.285297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16836f46-1ce8-406c-b8aa-d80a283dc6b2 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:21.286233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16836f46-1ce8-406c-b8aa-d80a283dc6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:21.286632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16836f46-1ce8-406c-b8aa-d80a283dc6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:21.308457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:21.324632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-16836f46-1ce8-406c-b8aa-d80a283dc6b2 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:21.327319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-16836f46-1ce8-406c-b8aa-d80a283dc6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:21.417874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 934/1867] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:21.423028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cc8b9-d50e-4eb1-af0a-8a722a9aac4b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:21.423654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 934/1868] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:21.424671 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cc8b9-d50e-4eb1-af0a-8a722a9aac4b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:21.425229 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cc8b9-d50e-4eb1-af0a-8a722a9aac4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:21.465001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-8f9cc8b9-d50e-4eb1-af0a-8a722a9aac4b None None] MFA Rules not processed for user `c3308ce8e3634d1693b34a912af386d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:21.466385 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8f9cc8b9-d50e-4eb1-af0a-8a722a9aac4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:21.548134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16836f46-1ce8-406c-b8aa-d80a283dc6b2 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:21.549583 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-16836f46-1ce8-406c-b8aa-d80a283dc6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:21.553867 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 935/1869] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:21.554903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9cc8b9-d50e-4eb1-af0a-8a722a9aac4b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:21.556461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8f9cc8b9-d50e-4eb1-af0a-8a722a9aac4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:21.615935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16836f46-1ce8-406c-b8aa-d80a283dc6b2 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:21.624922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c2291-210b-4703-8b37-a3e31e6edf8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:21.627625 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c2291-210b-4703-8b37-a3e31e6edf8f service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:21.628093 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c2291-210b-4703-8b37-a3e31e6edf8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:21.628517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c2291-210b-4703-8b37-a3e31e6edf8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:21.637924 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9cc8b9-d50e-4eb1-af0a-8a722a9aac4b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:21.643662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189b54d-6d37-4dcc-9850-e886a3634887 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:21.644424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189b54d-6d37-4dcc-9850-e886a3634887 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:21.644777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189b54d-6d37-4dcc-9850-e886a3634887 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:21.645164 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189b54d-6d37-4dcc-9850-e886a3634887 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:21.646376 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c2291-210b-4703-8b37-a3e31e6edf8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:21.648057 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4189b54d-6d37-4dcc-9850-e886a3634887 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:21.648985 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c2291-210b-4703-8b37-a3e31e6edf8f service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:21.665773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 935/1870] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:44:21.666821 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c2291-210b-4703-8b37-a3e31e6edf8f service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:21.668911 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7a6c2291-210b-4703-8b37-a3e31e6edf8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:21.698522 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c2291-210b-4703-8b37-a3e31e6edf8f service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:21.704565 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55cd05e-7dd5-4577-8f51-cfb4b16c3689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:21.705135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cd05e-7dd5-4577-8f51-cfb4b16c3689 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:21.705986 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cd05e-7dd5-4577-8f51-cfb4b16c3689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:21.706428 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55cd05e-7dd5-4577-8f51-cfb4b16c3689 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:21.707568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55cd05e-7dd5-4577-8f51-cfb4b16c3689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:21.710200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55cd05e-7dd5-4577-8f51-cfb4b16c3689 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:21.735373 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:21.751718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 936/1871] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:21 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:21.751718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55cd05e-7dd5-4577-8f51-cfb4b16c3689 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:21.753659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a55cd05e-7dd5-4577-8f51-cfb4b16c3689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:21.757869 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4189b54d-6d37-4dcc-9850-e886a3634887 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:21.759614 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4189b54d-6d37-4dcc-9850-e886a3634887 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:21.775120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 936/1872] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:21.776027 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189b54d-6d37-4dcc-9850-e886a3634887 service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:21.792114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189b54d-6d37-4dcc-9850-e886a3634887 service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:21.832653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55cd05e-7dd5-4577-8f51-cfb4b16c3689 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:21.837111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7639b-1275-4a59-b950-41a8b5d22db3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:21.838706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7639b-1275-4a59-b950-41a8b5d22db3 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:21.839398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7639b-1275-4a59-b950-41a8b5d22db3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:21.839832 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7639b-1275-4a59-b950-41a8b5d22db3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:21.874841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c1528-4a9a-44a9-8cea-06a668c62fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:21.875580 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c1528-4a9a-44a9-8cea-06a668c62fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:21.875869 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c1528-4a9a-44a9-8cea-06a668c62fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:21.876146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c1528-4a9a-44a9-8cea-06a668c62fc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:21.877008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c1528-4a9a-44a9-8cea-06a668c62fc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:21.879276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c1528-4a9a-44a9-8cea-06a668c62fc4 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:21.888795 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:21.917574 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 937/1873] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:21 2022] POST /identity/v3/users => generated 298 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:21.917855 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7639b-1275-4a59-b950-41a8b5d22db3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:21.919447 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7639b-1275-4a59-b950-41a8b5d22db3 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:21.921757 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c1528-4a9a-44a9-8cea-06a668c62fc4 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:21.933563 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 937/1874] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:21.944532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c1528-4a9a-44a9-8cea-06a668c62fc4 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:21.951289 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c9371-0cfd-49fb-8b4c-666f3836096b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:21.952128 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c9371-0cfd-49fb-8b4c-666f3836096b admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:21.952761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c9371-0cfd-49fb-8b4c-666f3836096b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:21.953138 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c9371-0cfd-49fb-8b4c-666f3836096b admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:21.954588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c9371-0cfd-49fb-8b4c-666f3836096b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:21.956365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c9371-0cfd-49fb-8b4c-666f3836096b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:21.973074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 938/1875] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:21.977120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7639b-1275-4a59-b950-41a8b5d22db3 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:21.989275 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7639b-1275-4a59-b950-41a8b5d22db3 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:21.996439 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89355222-b870-4ae1-a68b-6ce346e086f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:21.997825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89355222-b870-4ae1-a68b-6ce346e086f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:21.998165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89355222-b870-4ae1-a68b-6ce346e086f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:21.998511 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89355222-b870-4ae1-a68b-6ce346e086f5 admin admin] PATH_INFO: `/v3/projects/c65b224e6a8a46119d5566eca853b345/users/f2d5dac38cd14e3a8a1431d1382f7683/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:22.015744 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89355222-b870-4ae1-a68b-6ce346e086f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c65b224e6a8a46119d5566eca853b345, user_id=f2d5dac38cd14e3a8a1431d1382f7683, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:22.018924 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89355222-b870-4ae1-a68b-6ce346e086f5 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:22.064450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 938/1876] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:21 2022] PUT /identity/v3/projects/c65b224e6a8a46119d5566eca853b345/users/f2d5dac38cd14e3a8a1431d1382f7683/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:22.068602 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c9371-0cfd-49fb-8b4c-666f3836096b admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:22.088202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c9371-0cfd-49fb-8b4c-666f3836096b admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:22.168670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9208fe-d4e5-413f-8ac1-93e22184186e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:22.169529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9208fe-d4e5-413f-8ac1-93e22184186e admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:22.169957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9208fe-d4e5-413f-8ac1-93e22184186e admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:22.170323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9208fe-d4e5-413f-8ac1-93e22184186e admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:22.171225 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9208fe-d4e5-413f-8ac1-93e22184186e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:22.173442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9208fe-d4e5-413f-8ac1-93e22184186e admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:22.186145 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 939/1877] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:22.192122 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89355222-b870-4ae1-a68b-6ce346e086f5 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:22.210735 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89355222-b870-4ae1-a68b-6ce346e086f5 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:22.221132 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576b80d-8dc8-4e64-b720-a17dd99f3b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:22.222060 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576b80d-8dc8-4e64-b720-a17dd99f3b4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:22.222645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576b80d-8dc8-4e64-b720-a17dd99f3b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:22.222754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576b80d-8dc8-4e64-b720-a17dd99f3b4c admin admin] PATH_INFO: `/v3/projects/c65b224e6a8a46119d5566eca853b345/users/f2d5dac38cd14e3a8a1431d1382f7683/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:22.227865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576b80d-8dc8-4e64-b720-a17dd99f3b4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c65b224e6a8a46119d5566eca853b345, user_id=f2d5dac38cd14e3a8a1431d1382f7683, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:22.230838 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576b80d-8dc8-4e64-b720-a17dd99f3b4c admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:22.253763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 939/1878] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:22 2022] PUT /identity/v3/projects/c65b224e6a8a46119d5566eca853b345/users/f2d5dac38cd14e3a8a1431d1382f7683/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:22.265565 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b87cb4-2767-4441-a4a5-f45aac58a3a0 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:22.265860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b87cb4-2767-4441-a4a5-f45aac58a3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:22.266132 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b87cb4-2767-4441-a4a5-f45aac58a3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:22.304451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-c6b87cb4-2767-4441-a4a5-f45aac58a3a0 None None] MFA Rules not processed for user `f2d5dac38cd14e3a8a1431d1382f7683`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:22.305909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c6b87cb4-2767-4441-a4a5-f45aac58a3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:22.381922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576b80d-8dc8-4e64-b720-a17dd99f3b4c admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:22.385244 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4576b80d-8dc8-4e64-b720-a17dd99f3b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:22.387440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:22.444404 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 940/1879] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:22.459737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b87cb4-2767-4441-a4a5-f45aac58a3a0 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:22.466046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c6b87cb4-2767-4441-a4a5-f45aac58a3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:22.492793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576b80d-8dc8-4e64-b720-a17dd99f3b4c admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:22.500652 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6fb653-433f-49ef-8433-e81c7bce28e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:22.502395 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fb653-433f-49ef-8433-e81c7bce28e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:22.502926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fb653-433f-49ef-8433-e81c7bce28e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:22.503323 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6fb653-433f-49ef-8433-e81c7bce28e0 admin admin] PATH_INFO: `/v3/projects/61953f626c684582a7642fa6b9c0fc0c` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:22.505077 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6fb653-433f-49ef-8433-e81c7bce28e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61953f626c684582a7642fa6b9c0fc0c)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:22.507334 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6fb653-433f-49ef-8433-e81c7bce28e0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:22.580183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9b6fb653-433f-49ef-8433-e81c7bce28e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61953f626c684582a7642fa6b9c0fc0c', 'request_id': 'req-9b6fb653-433f-49ef-8433-e81c7bce28e0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:22.588715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9b6fb653-433f-49ef-8433-e81c7bce28e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61953f626c684582a7642fa6b9c0fc0c', 'request_id': 'req-9b6fb653-433f-49ef-8433-e81c7bce28e0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:22.596413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9b6fb653-433f-49ef-8433-e81c7bce28e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61953f626c684582a7642fa6b9c0fc0c', 'request_id': 'req-9b6fb653-433f-49ef-8433-e81c7bce28e0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:22.597148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b87cb4-2767-4441-a4a5-f45aac58a3a0 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:22.618827 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9b6fb653-433f-49ef-8433-e81c7bce28e0 admin admin] The token cache is being invalidate because project 61953f626c684582a7642fa6b9c0fc0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:22.619180 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9b6fb653-433f-49ef-8433-e81c7bce28e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:22.619990 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9b6fb653-433f-49ef-8433-e81c7bce28e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:22.622220 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 940/1880] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:22 2022] DELETE /identity/v3/projects/61953f626c684582a7642fa6b9c0fc0c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:22.625020 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6fb653-433f-49ef-8433-e81c7bce28e0 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:22.626677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9b6fb653-433f-49ef-8433-e81c7bce28e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:22.670599 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da90e8a-5668-42a5-836e-0fae128353a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:22.671561 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da90e8a-5668-42a5-836e-0fae128353a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:22.672144 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da90e8a-5668-42a5-836e-0fae128353a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:22.672595 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da90e8a-5668-42a5-836e-0fae128353a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:22.674685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-0da90e8a-5668-42a5-836e-0fae128353a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:22.683818 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:22.757563 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6fb653-433f-49ef-8433-e81c7bce28e0 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:22.764012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ae84e-1237-4440-be35-bf22287a905a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:22.765571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ae84e-1237-4440-be35-bf22287a905a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:22.766017 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ae84e-1237-4440-be35-bf22287a905a admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:22.766473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ae84e-1237-4440-be35-bf22287a905a admin admin] PATH_INFO: `/v3/projects/b2b42f77e8cd41f086eb703be44f849c` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:22.767992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ae84e-1237-4440-be35-bf22287a905a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2b42f77e8cd41f086eb703be44f849c)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:22.770404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ae84e-1237-4440-be35-bf22287a905a admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:22.776970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da90e8a-5668-42a5-836e-0fae128353a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:22.779768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da90e8a-5668-42a5-836e-0fae128353a3 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:22.793908 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 941/1881] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:22.795292 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da90e8a-5668-42a5-836e-0fae128353a3 service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:22.798179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-0da90e8a-5668-42a5-836e-0fae128353a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:22.814871 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da90e8a-5668-42a5-836e-0fae128353a3 service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:22.825906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f99ae84e-1237-4440-be35-bf22287a905a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2b42f77e8cd41f086eb703be44f849c', 'request_id': 'req-f99ae84e-1237-4440-be35-bf22287a905a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:22.834238 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f99ae84e-1237-4440-be35-bf22287a905a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2b42f77e8cd41f086eb703be44f849c', 'request_id': 'req-f99ae84e-1237-4440-be35-bf22287a905a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:22.841172 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f99ae84e-1237-4440-be35-bf22287a905a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2b42f77e8cd41f086eb703be44f849c', 'request_id': 'req-f99ae84e-1237-4440-be35-bf22287a905a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:22.859750 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f99ae84e-1237-4440-be35-bf22287a905a admin admin] The token cache is being invalidate because project b2b42f77e8cd41f086eb703be44f849c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:22.860201 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f99ae84e-1237-4440-be35-bf22287a905a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:22.861033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f99ae84e-1237-4440-be35-bf22287a905a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:22.863050 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 941/1882] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:22 2022] DELETE /identity/v3/projects/b2b42f77e8cd41f086eb703be44f849c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:22.866786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ae84e-1237-4440-be35-bf22287a905a admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:22.868682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f99ae84e-1237-4440-be35-bf22287a905a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:22.883359 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:22.896375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cbb93b-83d1-480b-8264-7baf0a117c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:22.897467 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbb93b-83d1-480b-8264-7baf0a117c8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:22.897811 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbb93b-83d1-480b-8264-7baf0a117c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:22.898093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbb93b-83d1-480b-8264-7baf0a117c8a admin admin] PATH_INFO: `/v3/users/199c008f282c400ea18e5ab2d434eacb` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:22.899459 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cbb93b-83d1-480b-8264-7baf0a117c8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=199c008f282c400ea18e5ab2d434eacb)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:22.901819 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cbb93b-83d1-480b-8264-7baf0a117c8a admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:22.951381 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ae84e-1237-4440-be35-bf22287a905a admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:22.957560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61389e47-01f5-47e4-982b-3819626e16a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:22.958720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61389e47-01f5-47e4-982b-3819626e16a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:22.959125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61389e47-01f5-47e4-982b-3819626e16a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:22.959325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61389e47-01f5-47e4-982b-3819626e16a9 admin admin] PATH_INFO: `/v3/users/8157a16f4059486e805fce80b582ebbb` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:22.960667 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61389e47-01f5-47e4-982b-3819626e16a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8157a16f4059486e805fce80b582ebbb)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:22.960951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a0cbb93b-83d1-480b-8264-7baf0a117c8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '199c008f282c400ea18e5ab2d434eacb'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:22.962867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61389e47-01f5-47e4-982b-3819626e16a9 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:22.975765 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a0cbb93b-83d1-480b-8264-7baf0a117c8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '199c008f282c400ea18e5ab2d434eacb'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:22.982678 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a0cbb93b-83d1-480b-8264-7baf0a117c8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '199c008f282c400ea18e5ab2d434eacb'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:22.983457 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a0cbb93b-83d1-480b-8264-7baf0a117c8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '199c008f282c400ea18e5ab2d434eacb'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:22.984068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a0cbb93b-83d1-480b-8264-7baf0a117c8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '199c008f282c400ea18e5ab2d434eacb'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:22.999634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 942/1883] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:22 2022] DELETE /identity/v3/users/199c008f282c400ea18e5ab2d434eacb => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:23.000445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cbb93b-83d1-480b-8264-7baf0a117c8a admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:23.001789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a0cbb93b-83d1-480b-8264-7baf0a117c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:23.017727 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-61389e47-01f5-47e4-982b-3819626e16a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8157a16f4059486e805fce80b582ebbb'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.031124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:23.036841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-61389e47-01f5-47e4-982b-3819626e16a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8157a16f4059486e805fce80b582ebbb'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.048119 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-61389e47-01f5-47e4-982b-3819626e16a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8157a16f4059486e805fce80b582ebbb'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.048955 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-61389e47-01f5-47e4-982b-3819626e16a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8157a16f4059486e805fce80b582ebbb'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.049952 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-61389e47-01f5-47e4-982b-3819626e16a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8157a16f4059486e805fce80b582ebbb'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.066609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 942/1884] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:22 2022] DELETE /identity/v3/users/8157a16f4059486e805fce80b582ebbb => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:23.080918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cbb93b-83d1-480b-8264-7baf0a117c8a admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:23.143971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89062d-b760-4f08-9895-b3b193a6740e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:23.144751 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89062d-b760-4f08-9895-b3b193a6740e service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:23.145146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89062d-b760-4f08-9895-b3b193a6740e service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:23.145490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89062d-b760-4f08-9895-b3b193a6740e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:23.148038 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-3b89062d-b760-4f08-9895-b3b193a6740e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:23.215441 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:23.239711 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89062d-b760-4f08-9895-b3b193a6740e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:23.241621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89062d-b760-4f08-9895-b3b193a6740e service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:23.255689 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 943/1885] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:23.327171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61389e47-01f5-47e4-982b-3819626e16a9 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:23.328226 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-61389e47-01f5-47e4-982b-3819626e16a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:23.434492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61389e47-01f5-47e4-982b-3819626e16a9 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:23.441444 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6cf68-e8a3-417c-af3d-112f88aa44c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:23.442469 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6cf68-e8a3-417c-af3d-112f88aa44c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:23.442830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6cf68-e8a3-417c-af3d-112f88aa44c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:23.443225 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6cf68-e8a3-417c-af3d-112f88aa44c6 admin admin] PATH_INFO: `/v3/projects/ed24d495cd4543bba2519c63dfb681ea` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:23.444534 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6cf68-e8a3-417c-af3d-112f88aa44c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed24d495cd4543bba2519c63dfb681ea)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:23.446916 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6cf68-e8a3-417c-af3d-112f88aa44c6 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:23.513932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-34e6cf68-e8a3-417c-af3d-112f88aa44c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed24d495cd4543bba2519c63dfb681ea', 'request_id': 'req-34e6cf68-e8a3-417c-af3d-112f88aa44c6'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.525132 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-34e6cf68-e8a3-417c-af3d-112f88aa44c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed24d495cd4543bba2519c63dfb681ea', 'request_id': 'req-34e6cf68-e8a3-417c-af3d-112f88aa44c6'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.526990 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89062d-b760-4f08-9895-b3b193a6740e service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:23.528869 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-3b89062d-b760-4f08-9895-b3b193a6740e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:23.535876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-34e6cf68-e8a3-417c-af3d-112f88aa44c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed24d495cd4543bba2519c63dfb681ea', 'request_id': 'req-34e6cf68-e8a3-417c-af3d-112f88aa44c6'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.559381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89062d-b760-4f08-9895-b3b193a6740e service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:23.571914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-34e6cf68-e8a3-417c-af3d-112f88aa44c6 admin admin] The token cache is being invalidate because project ed24d495cd4543bba2519c63dfb681ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:23.572220 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-34e6cf68-e8a3-417c-af3d-112f88aa44c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.573104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-34e6cf68-e8a3-417c-af3d-112f88aa44c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.588291 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 943/1886] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:23 2022] DELETE /identity/v3/projects/ed24d495cd4543bba2519c63dfb681ea => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:23.589899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780a81c-c8ba-464a-9200-4972b9f8f33f None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:23.590129 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780a81c-c8ba-464a-9200-4972b9f8f33f None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:23.591361 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780a81c-c8ba-464a-9200-4972b9f8f33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:23.615795 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-9780a81c-c8ba-464a-9200-4972b9f8f33f None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:23.617318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9780a81c-c8ba-464a-9200-4972b9f8f33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:23.634013 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f12ff9-7981-4b7b-9499-2c5d901009ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:23.634896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f12ff9-7981-4b7b-9499-2c5d901009ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:23.635229 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f12ff9-7981-4b7b-9499-2c5d901009ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:23.635509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f12ff9-7981-4b7b-9499-2c5d901009ad admin admin] PATH_INFO: `/v3/projects/ae5cfd2d297b45728ba0b7a99ac71278` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:23.637000 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f12ff9-7981-4b7b-9499-2c5d901009ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae5cfd2d297b45728ba0b7a99ac71278)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:23.639127 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f12ff9-7981-4b7b-9499-2c5d901009ad admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:23.688420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-97f12ff9-7981-4b7b-9499-2c5d901009ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae5cfd2d297b45728ba0b7a99ac71278', 'request_id': 'req-97f12ff9-7981-4b7b-9499-2c5d901009ad'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.690286 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 944/1887] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:23.691888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780a81c-c8ba-464a-9200-4972b9f8f33f None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:23.693270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9780a81c-c8ba-464a-9200-4972b9f8f33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:23.696955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-97f12ff9-7981-4b7b-9499-2c5d901009ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae5cfd2d297b45728ba0b7a99ac71278', 'request_id': 'req-97f12ff9-7981-4b7b-9499-2c5d901009ad'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.706382 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780a81c-c8ba-464a-9200-4972b9f8f33f None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:23.707733 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-97f12ff9-7981-4b7b-9499-2c5d901009ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae5cfd2d297b45728ba0b7a99ac71278', 'request_id': 'req-97f12ff9-7981-4b7b-9499-2c5d901009ad'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.711205 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:23.722301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-97f12ff9-7981-4b7b-9499-2c5d901009ad admin admin] The token cache is being invalidate because project ae5cfd2d297b45728ba0b7a99ac71278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:23.722758 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-97f12ff9-7981-4b7b-9499-2c5d901009ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.723752 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-97f12ff9-7981-4b7b-9499-2c5d901009ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.726251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 944/1888] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:23 2022] DELETE /identity/v3/projects/ae5cfd2d297b45728ba0b7a99ac71278 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:23.728444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fbaf3-0932-43d4-8cfc-90c04c6ad80c None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:23.728897 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fbaf3-0932-43d4-8cfc-90c04c6ad80c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:23.729367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fbaf3-0932-43d4-8cfc-90c04c6ad80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:23.754366 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-7f3fbaf3-0932-43d4-8cfc-90c04c6ad80c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:23.755606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-7f3fbaf3-0932-43d4-8cfc-90c04c6ad80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:23.802256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74c09c-6608-49ca-8889-02bd061b5945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:23.802812 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74c09c-6608-49ca-8889-02bd061b5945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:23.803131 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74c09c-6608-49ca-8889-02bd061b5945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:23.803461 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74c09c-6608-49ca-8889-02bd061b5945 admin admin] PATH_INFO: `/v3/users/c3308ce8e3634d1693b34a912af386d1` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:23.804871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74c09c-6608-49ca-8889-02bd061b5945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3308ce8e3634d1693b34a912af386d1)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:23.806798 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74c09c-6608-49ca-8889-02bd061b5945 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:23.834646 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 945/1889] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:23.836505 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd344-ba9f-4375-9234-409a8fe2ab6c None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:23.836960 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd344-ba9f-4375-9234-409a8fe2ab6c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:23.837399 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecd344-ba9f-4375-9234-409a8fe2ab6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:23.863469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-52ecd344-ba9f-4375-9234-409a8fe2ab6c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:23.864726 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-52ecd344-ba9f-4375-9234-409a8fe2ab6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:23.873386 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 946/1890] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:23.877120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecd344-ba9f-4375-9234-409a8fe2ab6c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:23.889245 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-ec74c09c-6608-49ca-8889-02bd061b5945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3308ce8e3634d1693b34a912af386d1'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.894417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecd344-ba9f-4375-9234-409a8fe2ab6c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:23.902345 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f95ed7-8cd5-4683-977f-f9c487f5d0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:23.903059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f95ed7-8cd5-4683-977f-f9c487f5d0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:23.903377 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f95ed7-8cd5-4683-977f-f9c487f5d0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:23.903644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f95ed7-8cd5-4683-977f-f9c487f5d0e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:23.904463 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f95ed7-8cd5-4683-977f-f9c487f5d0e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:44:23.904769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f95ed7-8cd5-4683-977f-f9c487f5d0e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:23.906413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f95ed7-8cd5-4683-977f-f9c487f5d0e9 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:23.908347 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:23.910310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-ec74c09c-6608-49ca-8889-02bd061b5945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3308ce8e3634d1693b34a912af386d1'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.923174 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 947/1891] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:44:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:44:23.924496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9f9f2-30d4-40ba-afc0-47d17f097de9 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:23.924948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9f9f2-30d4-40ba-afc0-47d17f097de9 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:23.925233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9f9f2-30d4-40ba-afc0-47d17f097de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:23.931355 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-ec74c09c-6608-49ca-8889-02bd061b5945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3308ce8e3634d1693b34a912af386d1'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.932620 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-ec74c09c-6608-49ca-8889-02bd061b5945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3308ce8e3634d1693b34a912af386d1'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.933639 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-ec74c09c-6608-49ca-8889-02bd061b5945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3308ce8e3634d1693b34a912af386d1'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:23.956679 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-02d9f9f2-30d4-40ba-afc0-47d17f097de9 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:23.957761 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 945/1892] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:23 2022] DELETE /identity/v3/users/c3308ce8e3634d1693b34a912af386d1 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:23.959583 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74c09c-6608-49ca-8889-02bd061b5945 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:23.961094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-02d9f9f2-30d4-40ba-afc0-47d17f097de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:23.961576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ec74c09c-6608-49ca-8889-02bd061b5945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:24.048728 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 948/1893] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:24.051248 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6652f-57b0-4b3f-accf-4021bc873110 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:24.051551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6652f-57b0-4b3f-accf-4021bc873110 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:24.051829 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6652f-57b0-4b3f-accf-4021bc873110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:24.059374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74c09c-6608-49ca-8889-02bd061b5945 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:24.067827 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091eee51-2b91-4fd7-a5b3-6595391bb65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:24.069196 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091eee51-2b91-4fd7-a5b3-6595391bb65b admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:24.069552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091eee51-2b91-4fd7-a5b3-6595391bb65b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:24.071368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091eee51-2b91-4fd7-a5b3-6595391bb65b admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:24.072362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091eee51-2b91-4fd7-a5b3-6595391bb65b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:24.076279 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091eee51-2b91-4fd7-a5b3-6595391bb65b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:24.088823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-2bd6652f-57b0-4b3f-accf-4021bc873110 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:24.090023 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2bd6652f-57b0-4b3f-accf-4021bc873110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:24.100160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 949/1894] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:24.101353 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6652f-57b0-4b3f-accf-4021bc873110 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:24.120476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6652f-57b0-4b3f-accf-4021bc873110 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:24.122703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 946/1895] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:23 2022] POST /identity/v3/projects => generated 363 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:24.123941 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091eee51-2b91-4fd7-a5b3-6595391bb65b admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:24.127310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-091eee51-2b91-4fd7-a5b3-6595391bb65b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:24.128870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56a874-68d8-4323-8722-81e4afc1cadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:24.130060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56a874-68d8-4323-8722-81e4afc1cadc admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:24.130060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56a874-68d8-4323-8722-81e4afc1cadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:24.130362 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56a874-68d8-4323-8722-81e4afc1cadc admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:24.131269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56a874-68d8-4323-8722-81e4afc1cadc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:44:24.132075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56a874-68d8-4323-8722-81e4afc1cadc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:24.133370 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56a874-68d8-4323-8722-81e4afc1cadc admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:24.151198 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 950/1896] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:44:24.151873 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:24.152095 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56a874-68d8-4323-8722-81e4afc1cadc admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:24.166812 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56a874-68d8-4323-8722-81e4afc1cadc admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:24.235290 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091eee51-2b91-4fd7-a5b3-6595391bb65b admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:24.238382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be02d8-8f9a-468b-8720-a4167002d5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:24.239186 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be02d8-8f9a-468b-8720-a4167002d5ca admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:24.239483 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be02d8-8f9a-468b-8720-a4167002d5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:24.239766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be02d8-8f9a-468b-8720-a4167002d5ca admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:24.240620 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be02d8-8f9a-468b-8720-a4167002d5ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:24.242487 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f650ee-5638-4d3a-9544-f79c3e1b77d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:24.243086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be02d8-8f9a-468b-8720-a4167002d5ca admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:24.244409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f650ee-5638-4d3a-9544-f79c3e1b77d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:24.244798 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f650ee-5638-4d3a-9544-f79c3e1b77d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:24.245418 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f650ee-5638-4d3a-9544-f79c3e1b77d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:24.260901 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f650ee-5638-4d3a-9544-f79c3e1b77d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:24.262659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f650ee-5638-4d3a-9544-f79c3e1b77d9 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:24.283413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 951/1897] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:24 2022] POST /identity/v3/users => generated 282 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:24.283413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be02d8-8f9a-468b-8720-a4167002d5ca admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:24.286752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 947/1898] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:24.287993 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f650ee-5638-4d3a-9544-f79c3e1b77d9 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:24.304481 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f650ee-5638-4d3a-9544-f79c3e1b77d9 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:24.306558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be02d8-8f9a-468b-8720-a4167002d5ca admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:24.312916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e2aa6-65e6-46e8-800f-c9041208d318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:24.313690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e2aa6-65e6-46e8-800f-c9041208d318 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:24.313995 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e2aa6-65e6-46e8-800f-c9041208d318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:24.314277 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e2aa6-65e6-46e8-800f-c9041208d318 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:24.315039 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e2aa6-65e6-46e8-800f-c9041208d318 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:24.317232 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e2aa6-65e6-46e8-800f-c9041208d318 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:24.329004 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b47e4e-3431-4888-8ffd-93c26e508240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:24.329727 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b47e4e-3431-4888-8ffd-93c26e508240 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:24.330237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b47e4e-3431-4888-8ffd-93c26e508240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:24.330237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b47e4e-3431-4888-8ffd-93c26e508240 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:24.332341 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b47e4e-3431-4888-8ffd-93c26e508240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:24.332689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b47e4e-3431-4888-8ffd-93c26e508240 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:24.351831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 948/1899] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:24.356477 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b47e4e-3431-4888-8ffd-93c26e508240 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:24.371757 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b47e4e-3431-4888-8ffd-93c26e508240 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:24.379394 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476d271-fa14-4b50-9cc8-55c4dc0fad52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:24.380189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476d271-fa14-4b50-9cc8-55c4dc0fad52 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:24.380951 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476d271-fa14-4b50-9cc8-55c4dc0fad52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:24.381062 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476d271-fa14-4b50-9cc8-55c4dc0fad52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:24.381861 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476d271-fa14-4b50-9cc8-55c4dc0fad52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:24.384818 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476d271-fa14-4b50-9cc8-55c4dc0fad52 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:24.388095 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 952/1900] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:24 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:24.392287 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e2aa6-65e6-46e8-800f-c9041208d318 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:24.400423 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 949/1901] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:44:24.406638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476d271-fa14-4b50-9cc8-55c4dc0fad52 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:24.425855 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476d271-fa14-4b50-9cc8-55c4dc0fad52 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:24.459973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:24.486982 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e2aa6-65e6-46e8-800f-c9041208d318 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:24.497276 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dd073-9f1f-42a2-8a69-5825afbb4f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:24.498233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd073-9f1f-42a2-8a69-5825afbb4f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:24.498518 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd073-9f1f-42a2-8a69-5825afbb4f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:24.499005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd073-9f1f-42a2-8a69-5825afbb4f4b admin admin] PATH_INFO: `/v3/projects/d11d62c5e61f4f07b4c061830348a595/users/344396919e764ec29482c7097874c5ed/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:24.499960 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09387f-1fe8-4886-a3d9-f9f41bbc2564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:24.501065 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09387f-1fe8-4886-a3d9-f9f41bbc2564 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:24.501449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09387f-1fe8-4886-a3d9-f9f41bbc2564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:24.501795 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09387f-1fe8-4886-a3d9-f9f41bbc2564 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:24.502918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09387f-1fe8-4886-a3d9-f9f41bbc2564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:24.505080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09387f-1fe8-4886-a3d9-f9f41bbc2564 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:24.525803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dd073-9f1f-42a2-8a69-5825afbb4f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d11d62c5e61f4f07b4c061830348a595, user_id=344396919e764ec29482c7097874c5ed, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:24.536405 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dd073-9f1f-42a2-8a69-5825afbb4f4b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:24.569789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 953/1902] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:24 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:44:24.574269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09387f-1fe8-4886-a3d9-f9f41bbc2564 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:24.589611 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 950/1903] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:24 2022] PUT /identity/v3/projects/d11d62c5e61f4f07b4c061830348a595/users/344396919e764ec29482c7097874c5ed/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:24.591561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9c712-169f-494e-8089-92eb75f1901a None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:24.591983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9c712-169f-494e-8089-92eb75f1901a None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:24.592369 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9c712-169f-494e-8089-92eb75f1901a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:24.604244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09387f-1fe8-4886-a3d9-f9f41bbc2564 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:24.625880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-8aa9c712-169f-494e-8089-92eb75f1901a None None] MFA Rules not processed for user `344396919e764ec29482c7097874c5ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:24.627485 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8aa9c712-169f-494e-8089-92eb75f1901a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:24.750466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee0798-d367-4ec3-be27-9173d9c4f648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:24.751280 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee0798-d367-4ec3-be27-9173d9c4f648 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:24.751793 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee0798-d367-4ec3-be27-9173d9c4f648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:24.752137 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee0798-d367-4ec3-be27-9173d9c4f648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:24.753165 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee0798-d367-4ec3-be27-9173d9c4f648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:24.755258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee0798-d367-4ec3-be27-9173d9c4f648 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:24.768405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 954/1904] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:24.769115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 951/1905] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:24 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:24.773502 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee0798-d367-4ec3-be27-9173d9c4f648 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:24.773606 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9c712-169f-494e-8089-92eb75f1901a None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:24.775143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1fee0798-d367-4ec3-be27-9173d9c4f648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:24.786958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9c712-169f-494e-8089-92eb75f1901a None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:24.791572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374b236-35f0-4ef7-ac72-5ce1f396bcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:24.792252 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374b236-35f0-4ef7-ac72-5ce1f396bcdd admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:24.792582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374b236-35f0-4ef7-ac72-5ce1f396bcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:24.792863 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374b236-35f0-4ef7-ac72-5ce1f396bcdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:24.793593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374b236-35f0-4ef7-ac72-5ce1f396bcdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:24.795325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374b236-35f0-4ef7-ac72-5ce1f396bcdd admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:24.799392 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:24.805799 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee0798-d367-4ec3-be27-9173d9c4f648 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:24.812955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb45446-9aa3-4007-85d1-06ef96f461d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:24.814416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb45446-9aa3-4007-85d1-06ef96f461d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:24.814621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 952/1906] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:24.815142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb45446-9aa3-4007-85d1-06ef96f461d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:24.815572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374b236-35f0-4ef7-ac72-5ce1f396bcdd admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:24.815835 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb45446-9aa3-4007-85d1-06ef96f461d4 admin admin] PATH_INFO: `/v3/projects/c0ef8176a9224b14b444ee66057cca20` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:24.818690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb45446-9aa3-4007-85d1-06ef96f461d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ef8176a9224b14b444ee66057cca20)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:24.820700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb45446-9aa3-4007-85d1-06ef96f461d4 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:24.828818 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374b236-35f0-4ef7-ac72-5ce1f396bcdd admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:24.834010 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffa6aa-6f0f-4b58-ad76-de6b1a57d4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:24.834946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffa6aa-6f0f-4b58-ad76-de6b1a57d4b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:24.835084 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffa6aa-6f0f-4b58-ad76-de6b1a57d4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:24.835300 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffa6aa-6f0f-4b58-ad76-de6b1a57d4b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:24.836000 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffa6aa-6f0f-4b58-ad76-de6b1a57d4b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:24.838131 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffa6aa-6f0f-4b58-ad76-de6b1a57d4b1 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:24.873892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 953/1907] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:24 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:24.873892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffa6aa-6f0f-4b58-ad76-de6b1a57d4b1 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:24.873892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-67ffa6aa-6f0f-4b58-ad76-de6b1a57d4b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:24.885476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-7eb45446-9aa3-4007-85d1-06ef96f461d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ef8176a9224b14b444ee66057cca20', 'request_id': 'req-7eb45446-9aa3-4007-85d1-06ef96f461d4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:24.897271 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-7eb45446-9aa3-4007-85d1-06ef96f461d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ef8176a9224b14b444ee66057cca20', 'request_id': 'req-7eb45446-9aa3-4007-85d1-06ef96f461d4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:24.911197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-7eb45446-9aa3-4007-85d1-06ef96f461d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ef8176a9224b14b444ee66057cca20', 'request_id': 'req-7eb45446-9aa3-4007-85d1-06ef96f461d4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:24.927848 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-7eb45446-9aa3-4007-85d1-06ef96f461d4 admin admin] The token cache is being invalidate because project c0ef8176a9224b14b444ee66057cca20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:24.928450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-7eb45446-9aa3-4007-85d1-06ef96f461d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:24.929387 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-7eb45446-9aa3-4007-85d1-06ef96f461d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:24.934453 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 955/1908] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:24 2022] DELETE /identity/v3/projects/c0ef8176a9224b14b444ee66057cca20 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:24.935658 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb45446-9aa3-4007-85d1-06ef96f461d4 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:24.937829 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-7eb45446-9aa3-4007-85d1-06ef96f461d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:24.948262 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffa6aa-6f0f-4b58-ad76-de6b1a57d4b1 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:24.955994 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0edf0-af66-465c-b332-1b4275f4c64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:24.957375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0edf0-af66-465c-b332-1b4275f4c64a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:24.957670 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0edf0-af66-465c-b332-1b4275f4c64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:24.958053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0edf0-af66-465c-b332-1b4275f4c64a admin admin] PATH_INFO: `/v3/users/2ad78f8b5fae4017a2b9630b37bb1ebf` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:24.960434 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce0edf0-af66-465c-b332-1b4275f4c64a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ad78f8b5fae4017a2b9630b37bb1ebf)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:24.964973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce0edf0-af66-465c-b332-1b4275f4c64a admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:25.008778 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:25.025517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb45446-9aa3-4007-85d1-06ef96f461d4 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:25.031422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-cce0edf0-af66-465c-b332-1b4275f4c64a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ad78f8b5fae4017a2b9630b37bb1ebf'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:25.033832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882082c-4f97-45e2-b546-852e960f7e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:25.035756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882082c-4f97-45e2-b546-852e960f7e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:25.036178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882082c-4f97-45e2-b546-852e960f7e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:25.036570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882082c-4f97-45e2-b546-852e960f7e5e admin admin] PATH_INFO: `/v3/projects/091eed2ae7314b22be894ac0e3b3c199/users/392acb39b08f4c1dbb30289fcab43990/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:25.051081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-cce0edf0-af66-465c-b332-1b4275f4c64a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ad78f8b5fae4017a2b9630b37bb1ebf'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:25.053685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882082c-4f97-45e2-b546-852e960f7e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=091eed2ae7314b22be894ac0e3b3c199, user_id=392acb39b08f4c1dbb30289fcab43990, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:25.056906 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882082c-4f97-45e2-b546-852e960f7e5e admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:25.065049 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-cce0edf0-af66-465c-b332-1b4275f4c64a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ad78f8b5fae4017a2b9630b37bb1ebf'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:25.065958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-cce0edf0-af66-465c-b332-1b4275f4c64a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ad78f8b5fae4017a2b9630b37bb1ebf'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:25.066722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-cce0edf0-af66-465c-b332-1b4275f4c64a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ad78f8b5fae4017a2b9630b37bb1ebf'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:25.076811 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 956/1909] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:24 2022] PUT /identity/v3/projects/091eed2ae7314b22be894ac0e3b3c199/users/392acb39b08f4c1dbb30289fcab43990/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:25.076811 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882082c-4f97-45e2-b546-852e960f7e5e admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:25.078979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-0882082c-4f97-45e2-b546-852e960f7e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:25.097257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 954/1910] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:24 2022] DELETE /identity/v3/users/2ad78f8b5fae4017a2b9630b37bb1ebf => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:25.101551 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0edf0-af66-465c-b332-1b4275f4c64a admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:25.103169 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-cce0edf0-af66-465c-b332-1b4275f4c64a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:25.166483 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882082c-4f97-45e2-b546-852e960f7e5e admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:25.195465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:25.211590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0edf0-af66-465c-b332-1b4275f4c64a admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:25.217475 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d682cf-65e2-4896-994a-aa59659127ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:25.218386 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d682cf-65e2-4896-994a-aa59659127ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:25.218669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d682cf-65e2-4896-994a-aa59659127ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:25.218948 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d682cf-65e2-4896-994a-aa59659127ac admin admin] PATH_INFO: `/v3/users/f2d5dac38cd14e3a8a1431d1382f7683` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:25.220204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d682cf-65e2-4896-994a-aa59659127ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2d5dac38cd14e3a8a1431d1382f7683)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:25.222192 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d682cf-65e2-4896-994a-aa59659127ac admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:25.236751 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714f9ea-646e-4460-bfb9-0018a4b1f10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:25.238057 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714f9ea-646e-4460-bfb9-0018a4b1f10c admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:25.238536 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714f9ea-646e-4460-bfb9-0018a4b1f10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:25.238991 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714f9ea-646e-4460-bfb9-0018a4b1f10c admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:25.239916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714f9ea-646e-4460-bfb9-0018a4b1f10c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:25.242428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714f9ea-646e-4460-bfb9-0018a4b1f10c admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:25.274761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 957/1911] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:25 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:25.276419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acdddc-73eb-4528-918d-2022ac7a59a1 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:25.276779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acdddc-73eb-4528-918d-2022ac7a59a1 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:25.277144 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acdddc-73eb-4528-918d-2022ac7a59a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:25.293584 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-94d682cf-65e2-4896-994a-aa59659127ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2d5dac38cd14e3a8a1431d1382f7683'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:25.311196 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-94d682cf-65e2-4896-994a-aa59659127ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2d5dac38cd14e3a8a1431d1382f7683'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:25.313036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-b7acdddc-73eb-4528-918d-2022ac7a59a1 None None] MFA Rules not processed for user `392acb39b08f4c1dbb30289fcab43990`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:25.314422 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b7acdddc-73eb-4528-918d-2022ac7a59a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:25.320824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-94d682cf-65e2-4896-994a-aa59659127ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2d5dac38cd14e3a8a1431d1382f7683'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:25.321828 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-94d682cf-65e2-4896-994a-aa59659127ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2d5dac38cd14e3a8a1431d1382f7683'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:25.322673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-94d682cf-65e2-4896-994a-aa59659127ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2d5dac38cd14e3a8a1431d1382f7683'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:25.337369 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 955/1912] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:25 2022] DELETE /identity/v3/users/f2d5dac38cd14e3a8a1431d1382f7683 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:25.338642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd97f06-9575-47e3-8c00-7b8be30f344a None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:25.338929 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd97f06-9575-47e3-8c00-7b8be30f344a None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:25.339266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd97f06-9575-47e3-8c00-7b8be30f344a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:25.358309 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-1fd97f06-9575-47e3-8c00-7b8be30f344a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:25.359306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-1fd97f06-9575-47e3-8c00-7b8be30f344a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:25.381560 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:25.391166 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 958/1913] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:25.393341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acdddc-73eb-4528-918d-2022ac7a59a1 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:25.394943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b7acdddc-73eb-4528-918d-2022ac7a59a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:25.410212 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acdddc-73eb-4528-918d-2022ac7a59a1 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:25.433579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 956/1914] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:25.434438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd97f06-9575-47e3-8c00-7b8be30f344a None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:25.435961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-1fd97f06-9575-47e3-8c00-7b8be30f344a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:25.456173 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd97f06-9575-47e3-8c00-7b8be30f344a None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:25.461950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072100db-738d-48ba-b185-cb29c039f339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:25.462656 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072100db-738d-48ba-b185-cb29c039f339 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:25.462944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072100db-738d-48ba-b185-cb29c039f339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:25.463265 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072100db-738d-48ba-b185-cb29c039f339 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:25.463969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072100db-738d-48ba-b185-cb29c039f339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:25.465935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072100db-738d-48ba-b185-cb29c039f339 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:25.472807 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca07c2-d631-4109-bb92-268f053dbabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:25.473491 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca07c2-d631-4109-bb92-268f053dbabe admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:25.473777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca07c2-d631-4109-bb92-268f053dbabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:25.474050 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca07c2-d631-4109-bb92-268f053dbabe admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:25.475247 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca07c2-d631-4109-bb92-268f053dbabe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:25.476908 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca07c2-d631-4109-bb92-268f053dbabe admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:25.487681 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 959/1915] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:25.488479 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca07c2-d631-4109-bb92-268f053dbabe admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:25.490202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-3dca07c2-d631-4109-bb92-268f053dbabe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:25.500017 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 957/1916] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:25 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:25.501041 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072100db-738d-48ba-b185-cb29c039f339 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:25.563487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca07c2-d631-4109-bb92-268f053dbabe admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:25.567964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:25.581779 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072100db-738d-48ba-b185-cb29c039f339 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:25.587410 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f15f016-ae86-4fbd-a87f-a9b2ec8627bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:25.588329 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15f016-ae86-4fbd-a87f-a9b2ec8627bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:25.588614 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15f016-ae86-4fbd-a87f-a9b2ec8627bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:25.588894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f15f016-ae86-4fbd-a87f-a9b2ec8627bc admin admin] PATH_INFO: `/v3/projects/4cb3beee5c5244ba83ed6c0edaca59b9/users/58ed2f0c28334a0fb6e712678a60561c/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:25.609430 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f15f016-ae86-4fbd-a87f-a9b2ec8627bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb3beee5c5244ba83ed6c0edaca59b9, user_id=58ed2f0c28334a0fb6e712678a60561c, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:25.614803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f15f016-ae86-4fbd-a87f-a9b2ec8627bc admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:25.618028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7717029-4d53-4479-9f90-d2769305661e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:25.618688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7717029-4d53-4479-9f90-d2769305661e service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:25.618970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7717029-4d53-4479-9f90-d2769305661e service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:25.619355 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7717029-4d53-4479-9f90-d2769305661e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:25.633514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7717029-4d53-4479-9f90-d2769305661e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:25.635031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7717029-4d53-4479-9f90-d2769305661e service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:25.637133 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 958/1917] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:25 2022] PUT /identity/v3/projects/4cb3beee5c5244ba83ed6c0edaca59b9/users/58ed2f0c28334a0fb6e712678a60561c/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:25.638301 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f15f016-ae86-4fbd-a87f-a9b2ec8627bc admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:25.649341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 960/1918] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:25.650262 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7717029-4d53-4479-9f90-d2769305661e service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:25.670696 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7717029-4d53-4479-9f90-d2769305661e service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:25.743556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:25.756815 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec30c6-d7f0-41ba-b678-fe1838173eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:25.757719 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec30c6-d7f0-41ba-b678-fe1838173eac admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:25.758008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec30c6-d7f0-41ba-b678-fe1838173eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:25.758419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec30c6-d7f0-41ba-b678-fe1838173eac admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:25.759304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec30c6-d7f0-41ba-b678-fe1838173eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:25.761125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec30c6-d7f0-41ba-b678-fe1838173eac admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:25.769260 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f15f016-ae86-4fbd-a87f-a9b2ec8627bc admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:25.775700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 961/1919] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:25.778015 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec30c6-d7f0-41ba-b678-fe1838173eac admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:25.779148 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5687030-f92a-4eab-accc-7871c7b8b0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:25.779911 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5687030-f92a-4eab-accc-7871c7b8b0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:25.780370 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-47ec30c6-d7f0-41ba-b678-fe1838173eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:25.781284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5687030-f92a-4eab-accc-7871c7b8b0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:25.781586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5687030-f92a-4eab-accc-7871c7b8b0a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:25.782301 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5687030-f92a-4eab-accc-7871c7b8b0a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:25.784284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5687030-f92a-4eab-accc-7871c7b8b0a1 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:25.796638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec30c6-d7f0-41ba-b678-fe1838173eac admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:25.803897 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687f406-f8a3-41c7-8f18-da3b61d12c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:25.804913 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687f406-f8a3-41c7-8f18-da3b61d12c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:25.805076 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687f406-f8a3-41c7-8f18-da3b61d12c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:25.805333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687f406-f8a3-41c7-8f18-da3b61d12c63 admin admin] PATH_INFO: `/v3/projects/c65b224e6a8a46119d5566eca853b345` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:25.807672 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f687f406-f8a3-41c7-8f18-da3b61d12c63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c65b224e6a8a46119d5566eca853b345)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:25.810060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f687f406-f8a3-41c7-8f18-da3b61d12c63 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:25.822749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 959/1920] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:25 2022] POST /identity/v3/users => generated 290 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:25.822749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5687030-f92a-4eab-accc-7871c7b8b0a1 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:25.836318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5687030-f92a-4eab-accc-7871c7b8b0a1 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:25.841664 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bcbf7e-d0fc-4415-8c54-9f69f0d0d43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:25.842576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcbf7e-d0fc-4415-8c54-9f69f0d0d43a admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:25.842862 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcbf7e-d0fc-4415-8c54-9f69f0d0d43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:25.843208 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcbf7e-d0fc-4415-8c54-9f69f0d0d43a admin admin] PATH_INFO: `/v3/projects/4cb3beee5c5244ba83ed6c0edaca59b9/users/58ed2f0c28334a0fb6e712678a60561c/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:25.845224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bcbf7e-d0fc-4415-8c54-9f69f0d0d43a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb3beee5c5244ba83ed6c0edaca59b9, user_id=58ed2f0c28334a0fb6e712678a60561c, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:25.847963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bcbf7e-d0fc-4415-8c54-9f69f0d0d43a admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:25.861537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 960/1921] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:25 2022] PUT /identity/v3/projects/4cb3beee5c5244ba83ed6c0edaca59b9/users/58ed2f0c28334a0fb6e712678a60561c/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:25.862395 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bcbf7e-d0fc-4415-8c54-9f69f0d0d43a admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:25.877273 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f687f406-f8a3-41c7-8f18-da3b61d12c63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c65b224e6a8a46119d5566eca853b345', 'request_id': 'req-f687f406-f8a3-41c7-8f18-da3b61d12c63'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:25.883587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f687f406-f8a3-41c7-8f18-da3b61d12c63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c65b224e6a8a46119d5566eca853b345', 'request_id': 'req-f687f406-f8a3-41c7-8f18-da3b61d12c63'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:25.892739 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f687f406-f8a3-41c7-8f18-da3b61d12c63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c65b224e6a8a46119d5566eca853b345', 'request_id': 'req-f687f406-f8a3-41c7-8f18-da3b61d12c63'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:25.906311 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f687f406-f8a3-41c7-8f18-da3b61d12c63 admin admin] The token cache is being invalidate because project c65b224e6a8a46119d5566eca853b345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:25.907266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f687f406-f8a3-41c7-8f18-da3b61d12c63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:25.907400 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f687f406-f8a3-41c7-8f18-da3b61d12c63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:25.909523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 962/1922] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:25 2022] DELETE /identity/v3/projects/c65b224e6a8a46119d5566eca853b345 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:25.911296 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab031831-f872-4679-b800-bbb96d8edd25 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:25.911608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab031831-f872-4679-b800-bbb96d8edd25 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:25.911980 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab031831-f872-4679-b800-bbb96d8edd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:25.937939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bcbf7e-d0fc-4415-8c54-9f69f0d0d43a admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:25.938951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-ab031831-f872-4679-b800-bbb96d8edd25 None None] MFA Rules not processed for user `58ed2f0c28334a0fb6e712678a60561c`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:25.940197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ab031831-f872-4679-b800-bbb96d8edd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:25.997326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:26.004732 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456a50e-1655-4b24-af70-2c82599932fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:26.005510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456a50e-1655-4b24-af70-2c82599932fd admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:26.006239 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456a50e-1655-4b24-af70-2c82599932fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:26.006867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456a50e-1655-4b24-af70-2c82599932fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:26.008463 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456a50e-1655-4b24-af70-2c82599932fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:26.010741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456a50e-1655-4b24-af70-2c82599932fd admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:26.014984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 963/1923] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:25 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:26.016551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d417b96-2c5d-47eb-a20f-510eb946e04a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:26.017718 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d417b96-2c5d-47eb-a20f-510eb946e04a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:26.018004 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d417b96-2c5d-47eb-a20f-510eb946e04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:26.029795 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 961/1924] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:26.032401 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456a50e-1655-4b24-af70-2c82599932fd admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:26.035528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d456a50e-1655-4b24-af70-2c82599932fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:26.038481 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-2d417b96-2c5d-47eb-a20f-510eb946e04a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:26.039685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2d417b96-2c5d-47eb-a20f-510eb946e04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:26.048165 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 964/1925] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:26.048995 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d417b96-2c5d-47eb-a20f-510eb946e04a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:26.049690 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456a50e-1655-4b24-af70-2c82599932fd admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:26.050247 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2d417b96-2c5d-47eb-a20f-510eb946e04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:26.060042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401be962-61d9-4fe7-9847-ff1b00a15ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:26.061054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401be962-61d9-4fe7-9847-ff1b00a15ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:26.061335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401be962-61d9-4fe7-9847-ff1b00a15ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:26.061634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401be962-61d9-4fe7-9847-ff1b00a15ba7 admin admin] PATH_INFO: `/v3/projects/2caf329bfe474924b19ffbfc46d90205` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:26.062824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401be962-61d9-4fe7-9847-ff1b00a15ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2caf329bfe474924b19ffbfc46d90205)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:26.064846 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401be962-61d9-4fe7-9847-ff1b00a15ba7 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:26.113713 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-401be962-61d9-4fe7-9847-ff1b00a15ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2caf329bfe474924b19ffbfc46d90205', 'request_id': 'req-401be962-61d9-4fe7-9847-ff1b00a15ba7'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:26.114546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d417b96-2c5d-47eb-a20f-510eb946e04a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:26.120144 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:26.121417 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-401be962-61d9-4fe7-9847-ff1b00a15ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2caf329bfe474924b19ffbfc46d90205', 'request_id': 'req-401be962-61d9-4fe7-9847-ff1b00a15ba7'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:26.127471 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-401be962-61d9-4fe7-9847-ff1b00a15ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2caf329bfe474924b19ffbfc46d90205', 'request_id': 'req-401be962-61d9-4fe7-9847-ff1b00a15ba7'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:26.141714 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-401be962-61d9-4fe7-9847-ff1b00a15ba7 admin admin] The token cache is being invalidate because project 2caf329bfe474924b19ffbfc46d90205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:26.141990 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-401be962-61d9-4fe7-9847-ff1b00a15ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:26.142572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-401be962-61d9-4fe7-9847-ff1b00a15ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:26.144208 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 962/1926] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:26 2022] DELETE /identity/v3/projects/2caf329bfe474924b19ffbfc46d90205 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:26.147170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401be962-61d9-4fe7-9847-ff1b00a15ba7 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:26.147170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-401be962-61d9-4fe7-9847-ff1b00a15ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:26.170677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cebf30-829c-48b6-aaed-10fd87b2a0f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:26.171374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cebf30-829c-48b6-aaed-10fd87b2a0f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:26.171658 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cebf30-829c-48b6-aaed-10fd87b2a0f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:26.171943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cebf30-829c-48b6-aaed-10fd87b2a0f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:26.173719 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d2cebf30-829c-48b6-aaed-10fd87b2a0f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:26.208666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401be962-61d9-4fe7-9847-ff1b00a15ba7 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:26.213656 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea3e94-2b0a-49a1-a0f0-f95836134eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:26.214332 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea3e94-2b0a-49a1-a0f0-f95836134eae admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:26.214613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea3e94-2b0a-49a1-a0f0-f95836134eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:26.214891 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea3e94-2b0a-49a1-a0f0-f95836134eae admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:26.215651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea3e94-2b0a-49a1-a0f0-f95836134eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:26.217350 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea3e94-2b0a-49a1-a0f0-f95836134eae admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:26.227682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 963/1927] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:26.228907 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258207c-71e6-44a0-9063-ffc382d43d47 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:26.229195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258207c-71e6-44a0-9063-ffc382d43d47 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:26.229475 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258207c-71e6-44a0-9063-ffc382d43d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:26.238376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cebf30-829c-48b6-aaed-10fd87b2a0f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:26.239905 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cebf30-829c-48b6-aaed-10fd87b2a0f3 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:26.248430 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-6258207c-71e6-44a0-9063-ffc382d43d47 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:26.249690 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-6258207c-71e6-44a0-9063-ffc382d43d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:26.249834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 965/1928] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:26 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:26.250638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cebf30-829c-48b6-aaed-10fd87b2a0f3 service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:26.251996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d2cebf30-829c-48b6-aaed-10fd87b2a0f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:26.263602 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 964/1929] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:26.265076 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258207c-71e6-44a0-9063-ffc382d43d47 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:26.268820 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cebf30-829c-48b6-aaed-10fd87b2a0f3 service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:26.275485 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6258207c-71e6-44a0-9063-ffc382d43d47 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:26.275837 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d06354-1edb-4f91-8534-1dd053f0e154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:26.276757 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d06354-1edb-4f91-8534-1dd053f0e154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:26.277102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d06354-1edb-4f91-8534-1dd053f0e154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:26.277359 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d06354-1edb-4f91-8534-1dd053f0e154 admin admin] PATH_INFO: `/v3/users/56ea498768154019a9a0c829e8673065` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:26.278665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d06354-1edb-4f91-8534-1dd053f0e154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56ea498768154019a9a0c829e8673065)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:26.280750 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d06354-1edb-4f91-8534-1dd053f0e154 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:26.281310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f0452-d589-4dc9-97a4-c04f4436303c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:26.282242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f0452-d589-4dc9-97a4-c04f4436303c admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:26.282687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f0452-d589-4dc9-97a4-c04f4436303c admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:26.283042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f0452-d589-4dc9-97a4-c04f4436303c admin admin] PATH_INFO: `/v3/projects/e24ef305d5c9427fbba9c9da6c56d4c0/users/1fd72a6c64cc4db3bcab684da0421581/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:26.301854 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f0452-d589-4dc9-97a4-c04f4436303c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e24ef305d5c9427fbba9c9da6c56d4c0, user_id=1fd72a6c64cc4db3bcab684da0421581, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:26.305267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f0452-d589-4dc9-97a4-c04f4436303c admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:26.323455 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 965/1930] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:26 2022] PUT /identity/v3/projects/e24ef305d5c9427fbba9c9da6c56d4c0/users/1fd72a6c64cc4db3bcab684da0421581/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:26.324350 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f0452-d589-4dc9-97a4-c04f4436303c admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:26.347954 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c5d06354-1edb-4f91-8534-1dd053f0e154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56ea498768154019a9a0c829e8673065'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:26.365217 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c5d06354-1edb-4f91-8534-1dd053f0e154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56ea498768154019a9a0c829e8673065'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:26.388600 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c5d06354-1edb-4f91-8534-1dd053f0e154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56ea498768154019a9a0c829e8673065'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:26.389644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c5d06354-1edb-4f91-8534-1dd053f0e154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56ea498768154019a9a0c829e8673065'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:26.390551 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:26.391845 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c5d06354-1edb-4f91-8534-1dd053f0e154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56ea498768154019a9a0c829e8673065'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:26.410946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f0452-d589-4dc9-97a4-c04f4436303c admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:26.417126 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc315b-7d01-440c-8e14-3c5e6ea6ac6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:26.417860 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc315b-7d01-440c-8e14-3c5e6ea6ac6a admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:26.418148 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc315b-7d01-440c-8e14-3c5e6ea6ac6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:26.418475 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc315b-7d01-440c-8e14-3c5e6ea6ac6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:26.419343 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc315b-7d01-440c-8e14-3c5e6ea6ac6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:44:26.419707 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc315b-7d01-440c-8e14-3c5e6ea6ac6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:26.421537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc315b-7d01-440c-8e14-3c5e6ea6ac6a admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:26.424475 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 966/1931] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:26 2022] DELETE /identity/v3/users/56ea498768154019a9a0c829e8673065 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:26.425906 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824be88-6a7e-473d-860a-64d435ee911d None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:26.426259 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824be88-6a7e-473d-860a-64d435ee911d None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:26.426549 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824be88-6a7e-473d-860a-64d435ee911d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:26.435647 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 966/1932] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:44:26.439841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc315b-7d01-440c-8e14-3c5e6ea6ac6a admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:26.442160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-78dc315b-7d01-440c-8e14-3c5e6ea6ac6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:26.459795 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc315b-7d01-440c-8e14-3c5e6ea6ac6a admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:26.482468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-6824be88-6a7e-473d-860a-64d435ee911d None None] MFA Rules not processed for user `1fd72a6c64cc4db3bcab684da0421581`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:26.484588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6824be88-6a7e-473d-860a-64d435ee911d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:26.540669 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:26.566534 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e05c7-1d60-485c-b75d-072891f2a2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:26.569307 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e05c7-1d60-485c-b75d-072891f2a2b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:26.571111 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e05c7-1d60-485c-b75d-072891f2a2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:26.571414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e05c7-1d60-485c-b75d-072891f2a2b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:26.574271 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e05c7-1d60-485c-b75d-072891f2a2b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:26.577652 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e05c7-1d60-485c-b75d-072891f2a2b5 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:26.651868 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 967/1933] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:26.659517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824be88-6a7e-473d-860a-64d435ee911d None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:26.659517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6824be88-6a7e-473d-860a-64d435ee911d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:26.680103 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 967/1934] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:26 2022] POST /identity/v3/projects => generated 361 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:44:26.682236 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e05c7-1d60-485c-b75d-072891f2a2b5 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:26.686429 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824be88-6a7e-473d-860a-64d435ee911d None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:26.783676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261ec04-13b8-4afb-a41a-b63d40b364a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:26.784642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261ec04-13b8-4afb-a41a-b63d40b364a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:26.785365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261ec04-13b8-4afb-a41a-b63d40b364a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:26.785693 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261ec04-13b8-4afb-a41a-b63d40b364a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:26.786825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e05c7-1d60-485c-b75d-072891f2a2b5 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:26.786962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261ec04-13b8-4afb-a41a-b63d40b364a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:26.789013 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261ec04-13b8-4afb-a41a-b63d40b364a1 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:26.792354 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3aab36-aa53-4be6-984c-7b1d30947ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:26.793061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3aab36-aa53-4be6-984c-7b1d30947ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:26.793329 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3aab36-aa53-4be6-984c-7b1d30947ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:26.793608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3aab36-aa53-4be6-984c-7b1d30947ca5 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:26.794488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3aab36-aa53-4be6-984c-7b1d30947ca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:26.811908 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3aab36-aa53-4be6-984c-7b1d30947ca5 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:26.828158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 968/1935] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:26 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:26.828158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261ec04-13b8-4afb-a41a-b63d40b364a1 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:26.829972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d261ec04-13b8-4afb-a41a-b63d40b364a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:26.855407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 968/1936] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:26 2022] POST /identity/v3/users => generated 281 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:26.855407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3aab36-aa53-4be6-984c-7b1d30947ca5 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:26.866144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3aab36-aa53-4be6-984c-7b1d30947ca5 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:26.925528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:26.930129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261ec04-13b8-4afb-a41a-b63d40b364a1 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:26.935174 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ed433-2e46-4fbf-8321-02bde6cfe6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:26.936087 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ed433-2e46-4fbf-8321-02bde6cfe6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:26.936376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ed433-2e46-4fbf-8321-02bde6cfe6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:26.936692 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ed433-2e46-4fbf-8321-02bde6cfe6ae admin admin] PATH_INFO: `/v3/projects/9f6945d7052f4833987fa295ec0c06fd` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:26.938028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ed433-2e46-4fbf-8321-02bde6cfe6ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f6945d7052f4833987fa295ec0c06fd)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:26.940038 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ed433-2e46-4fbf-8321-02bde6cfe6ae admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:26.940411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a50f9-455f-466a-8426-63ca742663a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:26.941135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a50f9-455f-466a-8426-63ca742663a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:26.941426 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a50f9-455f-466a-8426-63ca742663a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:26.941701 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a50f9-455f-466a-8426-63ca742663a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:26.942526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a50f9-455f-466a-8426-63ca742663a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:26.944531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a50f9-455f-466a-8426-63ca742663a5 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:26.970683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 969/1937] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:26 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:26.970683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a50f9-455f-466a-8426-63ca742663a5 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:26.980344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a50f9-455f-466a-8426-63ca742663a5 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:26.992813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:26.995187 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-832ed433-2e46-4fbf-8321-02bde6cfe6ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f6945d7052f4833987fa295ec0c06fd', 'request_id': 'req-832ed433-2e46-4fbf-8321-02bde6cfe6ae'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.002053 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-832ed433-2e46-4fbf-8321-02bde6cfe6ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f6945d7052f4833987fa295ec0c06fd', 'request_id': 'req-832ed433-2e46-4fbf-8321-02bde6cfe6ae'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.012294 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-832ed433-2e46-4fbf-8321-02bde6cfe6ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f6945d7052f4833987fa295ec0c06fd', 'request_id': 'req-832ed433-2e46-4fbf-8321-02bde6cfe6ae'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.024829 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-832ed433-2e46-4fbf-8321-02bde6cfe6ae admin admin] The token cache is being invalidate because project 9f6945d7052f4833987fa295ec0c06fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:27.025186 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-832ed433-2e46-4fbf-8321-02bde6cfe6ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.025926 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-832ed433-2e46-4fbf-8321-02bde6cfe6ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.030564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 969/1938] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:26 2022] DELETE /identity/v3/projects/9f6945d7052f4833987fa295ec0c06fd => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:27.030564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ed433-2e46-4fbf-8321-02bde6cfe6ae admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:27.030564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-832ed433-2e46-4fbf-8321-02bde6cfe6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:27.047698 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07163dd3-e46b-4f76-8c89-dbe161232f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:27.048392 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07163dd3-e46b-4f76-8c89-dbe161232f77 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:27.048679 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07163dd3-e46b-4f76-8c89-dbe161232f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:27.049235 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07163dd3-e46b-4f76-8c89-dbe161232f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:27.049691 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07163dd3-e46b-4f76-8c89-dbe161232f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:27.051362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07163dd3-e46b-4f76-8c89-dbe161232f77 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:27.064208 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 970/1939] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:27.064208 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07163dd3-e46b-4f76-8c89-dbe161232f77 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:27.064208 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-07163dd3-e46b-4f76-8c89-dbe161232f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:27.075990 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07163dd3-e46b-4f76-8c89-dbe161232f77 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:27.080813 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea620508-3c5d-4825-90ac-b1628321b163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:27.081472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea620508-3c5d-4825-90ac-b1628321b163 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:27.081757 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea620508-3c5d-4825-90ac-b1628321b163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:27.082042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea620508-3c5d-4825-90ac-b1628321b163 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:27.082765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea620508-3c5d-4825-90ac-b1628321b163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:27.084403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea620508-3c5d-4825-90ac-b1628321b163 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:27.094464 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 971/1940] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:27.096209 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ed433-2e46-4fbf-8321-02bde6cfe6ae admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:27.097299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea620508-3c5d-4825-90ac-b1628321b163 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:27.098956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ea620508-3c5d-4825-90ac-b1628321b163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:27.100997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4625487-357a-4345-8daa-5d8d3b97309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:27.101961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4625487-357a-4345-8daa-5d8d3b97309b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:27.102328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4625487-357a-4345-8daa-5d8d3b97309b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:27.102674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4625487-357a-4345-8daa-5d8d3b97309b admin admin] PATH_INFO: `/v3/users/344396919e764ec29482c7097874c5ed` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:27.104003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4625487-357a-4345-8daa-5d8d3b97309b admin admin] RBAC: Authorizing `identity:delete_user(user_id=344396919e764ec29482c7097874c5ed)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:27.105959 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4625487-357a-4345-8daa-5d8d3b97309b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:27.112062 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea620508-3c5d-4825-90ac-b1628321b163 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:27.117090 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0bb93-6227-4c7e-a306-7cf44807ad6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:27.117939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0bb93-6227-4c7e-a306-7cf44807ad6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:27.118220 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0bb93-6227-4c7e-a306-7cf44807ad6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:27.118507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0bb93-6227-4c7e-a306-7cf44807ad6d admin admin] PATH_INFO: `/v3/users/8bc2fb0f5f75423987af604635563d8b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:27.119803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0bb93-6227-4c7e-a306-7cf44807ad6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bc2fb0f5f75423987af604635563d8b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:27.121711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0bb93-6227-4c7e-a306-7cf44807ad6d admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:27.158760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c4625487-357a-4345-8daa-5d8d3b97309b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '344396919e764ec29482c7097874c5ed'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.171213 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c4625487-357a-4345-8daa-5d8d3b97309b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '344396919e764ec29482c7097874c5ed'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.174933 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-5ce0bb93-6227-4c7e-a306-7cf44807ad6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bc2fb0f5f75423987af604635563d8b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.177985 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c4625487-357a-4345-8daa-5d8d3b97309b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '344396919e764ec29482c7097874c5ed'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.178678 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c4625487-357a-4345-8daa-5d8d3b97309b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '344396919e764ec29482c7097874c5ed'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.179325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c4625487-357a-4345-8daa-5d8d3b97309b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '344396919e764ec29482c7097874c5ed'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.187737 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-5ce0bb93-6227-4c7e-a306-7cf44807ad6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bc2fb0f5f75423987af604635563d8b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.193194 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 970/1941] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:27 2022] DELETE /identity/v3/users/344396919e764ec29482c7097874c5ed => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:27.194119 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4625487-357a-4345-8daa-5d8d3b97309b admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:27.195489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c4625487-357a-4345-8daa-5d8d3b97309b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:27.196381 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-5ce0bb93-6227-4c7e-a306-7cf44807ad6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bc2fb0f5f75423987af604635563d8b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.197080 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-5ce0bb93-6227-4c7e-a306-7cf44807ad6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bc2fb0f5f75423987af604635563d8b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.197757 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-5ce0bb93-6227-4c7e-a306-7cf44807ad6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bc2fb0f5f75423987af604635563d8b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.212097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 972/1942] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:27 2022] DELETE /identity/v3/users/8bc2fb0f5f75423987af604635563d8b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:27.213127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0bb93-6227-4c7e-a306-7cf44807ad6d admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:27.214731 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5ce0bb93-6227-4c7e-a306-7cf44807ad6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:27.262339 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:27.264439 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:27.271650 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4625487-357a-4345-8daa-5d8d3b97309b admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:27.292464 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0bb93-6227-4c7e-a306-7cf44807ad6d admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:27.297836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c36b04-c7a5-4061-966a-efc561bee062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:27.299044 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c36b04-c7a5-4061-966a-efc561bee062 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:27.299357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c36b04-c7a5-4061-966a-efc561bee062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:27.299717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c36b04-c7a5-4061-966a-efc561bee062 admin admin] PATH_INFO: `/v3/projects/55fa311fd61446499bf25fdc96cf9f7c/users/93905ff928f04bafa54542635ddab9eb/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:27.315893 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c36b04-c7a5-4061-966a-efc561bee062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55fa311fd61446499bf25fdc96cf9f7c, user_id=93905ff928f04bafa54542635ddab9eb, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:27.319105 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c36b04-c7a5-4061-966a-efc561bee062 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:27.334739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 973/1943] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:27 2022] PUT /identity/v3/projects/55fa311fd61446499bf25fdc96cf9f7c/users/93905ff928f04bafa54542635ddab9eb/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:27.336630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010d481-119e-461c-bf4e-1d7875031167 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:27.336932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010d481-119e-461c-bf4e-1d7875031167 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:27.337210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010d481-119e-461c-bf4e-1d7875031167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:27.340712 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c25b57-1b5b-4676-8983-0751505950d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:27.341499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c25b57-1b5b-4676-8983-0751505950d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:27.341747 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c25b57-1b5b-4676-8983-0751505950d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:27.342028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c25b57-1b5b-4676-8983-0751505950d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:27.342764 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c25b57-1b5b-4676-8983-0751505950d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:27.344632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c25b57-1b5b-4676-8983-0751505950d8 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:27.355719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-3010d481-119e-461c-bf4e-1d7875031167 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:27.357129 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3010d481-119e-461c-bf4e-1d7875031167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:27.357691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 971/1944] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:27.358501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c25b57-1b5b-4676-8983-0751505950d8 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:27.372219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c25b57-1b5b-4676-8983-0751505950d8 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:27.439177 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 974/1945] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:27.440159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010d481-119e-461c-bf4e-1d7875031167 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:27.443892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3010d481-119e-461c-bf4e-1d7875031167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:27.446334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87160d-9081-4627-a896-7ad27ca52b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:27.447134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87160d-9081-4627-a896-7ad27ca52b6e admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:27.447428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87160d-9081-4627-a896-7ad27ca52b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:27.447707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87160d-9081-4627-a896-7ad27ca52b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:27.448578 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c87160d-9081-4627-a896-7ad27ca52b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:27.450381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c87160d-9081-4627-a896-7ad27ca52b6e admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:27.463517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 972/1946] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:27.468116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87160d-9081-4627-a896-7ad27ca52b6e admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:27.469843 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010d481-119e-461c-bf4e-1d7875031167 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:27.469968 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1c87160d-9081-4627-a896-7ad27ca52b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:27.478273 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc550dc0-a26a-4db0-a48d-e7af5fb92bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:27.480501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc550dc0-a26a-4db0-a48d-e7af5fb92bb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:27.480966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc550dc0-a26a-4db0-a48d-e7af5fb92bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:27.481622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc550dc0-a26a-4db0-a48d-e7af5fb92bb0 admin admin] PATH_INFO: `/v3/projects/3b7d57b0794c48e2826302b1c58232bf/users/d25651f351294c6ab30291b0a04b149e/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:27.506963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc550dc0-a26a-4db0-a48d-e7af5fb92bb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b7d57b0794c48e2826302b1c58232bf, user_id=d25651f351294c6ab30291b0a04b149e, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:27.511270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc550dc0-a26a-4db0-a48d-e7af5fb92bb0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:27.539161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 975/1947] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:27 2022] PUT /identity/v3/projects/3b7d57b0794c48e2826302b1c58232bf/users/d25651f351294c6ab30291b0a04b149e/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:27.540926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc550dc0-a26a-4db0-a48d-e7af5fb92bb0 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:27.565998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87160d-9081-4627-a896-7ad27ca52b6e admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:27.621827 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc550dc0-a26a-4db0-a48d-e7af5fb92bb0 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:27.626578 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf74a0-a7a9-4c05-8459-deedeed87cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:27.627233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04630e05-5aa4-46e9-b9d4-59fc6e9df660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:27.627586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf74a0-a7a9-4c05-8459-deedeed87cba admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:27.627865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf74a0-a7a9-4c05-8459-deedeed87cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:27.627958 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04630e05-5aa4-46e9-b9d4-59fc6e9df660 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:27.628136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf74a0-a7a9-4c05-8459-deedeed87cba admin admin] PATH_INFO: `/v3/projects/55fa311fd61446499bf25fdc96cf9f7c/users/93905ff928f04bafa54542635ddab9eb/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:27.628240 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04630e05-5aa4-46e9-b9d4-59fc6e9df660 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:27.628423 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04630e05-5aa4-46e9-b9d4-59fc6e9df660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:27.630095 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf74a0-a7a9-4c05-8459-deedeed87cba admin admin] RBAC: Authorizing `identity:create_grant(project_id=55fa311fd61446499bf25fdc96cf9f7c, user_id=93905ff928f04bafa54542635ddab9eb, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:27.632833 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf74a0-a7a9-4c05-8459-deedeed87cba admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:27.634631 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:27.639561 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04630e05-5aa4-46e9-b9d4-59fc6e9df660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:27.641065 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04630e05-5aa4-46e9-b9d4-59fc6e9df660 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:27.643590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 976/1948] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:27 2022] PUT /identity/v3/projects/55fa311fd61446499bf25fdc96cf9f7c/users/93905ff928f04bafa54542635ddab9eb/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:27.644398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf74a0-a7a9-4c05-8459-deedeed87cba admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:27.645912 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-dfdf74a0-a7a9-4c05-8459-deedeed87cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:27.660276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 973/1949] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:27.661637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd0816-b632-42ea-8f9e-673c0805b5e7 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:27.661984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd0816-b632-42ea-8f9e-673c0805b5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:27.662483 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd0816-b632-42ea-8f9e-673c0805b5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:27.691707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-5fbd0816-b632-42ea-8f9e-673c0805b5e7 None None] MFA Rules not processed for user `d25651f351294c6ab30291b0a04b149e`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:27.692723 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:27.693373 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5fbd0816-b632-42ea-8f9e-673c0805b5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:27.765962 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf74a0-a7a9-4c05-8459-deedeed87cba admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:27.773845 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:27.775100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:27.775410 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:27.775761 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1 admin admin] PATH_INFO: `/v3/projects/31afd2ad2f604a3c9f74cfd68f4c9172` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:27.777193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31afd2ad2f604a3c9f74cfd68f4c9172)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:27.780018 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:27.808443 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 974/1950] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:27 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:27.810492 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb791d-1d03-4402-b7e3-fe3100ac0126 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:27.811099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb791d-1d03-4402-b7e3-fe3100ac0126 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:27.812858 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb791d-1d03-4402-b7e3-fe3100ac0126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:27.870114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-aaeb791d-1d03-4402-b7e3-fe3100ac0126 None None] MFA Rules not processed for user `93905ff928f04bafa54542635ddab9eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:27.873262 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-aaeb791d-1d03-4402-b7e3-fe3100ac0126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:27.879403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31afd2ad2f604a3c9f74cfd68f4c9172', 'request_id': 'req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.887004 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31afd2ad2f604a3c9f74cfd68f4c9172', 'request_id': 'req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.895155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31afd2ad2f604a3c9f74cfd68f4c9172', 'request_id': 'req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.913312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1 admin admin] The token cache is being invalidate because project 31afd2ad2f604a3c9f74cfd68f4c9172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:27.914064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.915331 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:27.925108 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 977/1951] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:27 2022] DELETE /identity/v3/projects/31afd2ad2f604a3c9f74cfd68f4c9172 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:27.926386 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:27.928090 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:27.973146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 975/1952] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:27.975446 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce19998-5138-41ca-a8ec-1efe0c13eb48 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:27.976018 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce19998-5138-41ca-a8ec-1efe0c13eb48 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:27.976429 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce19998-5138-41ca-a8ec-1efe0c13eb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:28.011647 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-4ce19998-5138-41ca-a8ec-1efe0c13eb48 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:28.013078 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4ce19998-5138-41ca-a8ec-1efe0c13eb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:28.035927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c0c40-b3ac-473c-b8a3-c3dbcec878c1 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:28.041852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb66908-f8b4-4e3d-b91d-2b318abf8b3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:28.042540 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb66908-f8b4-4e3d-b91d-2b318abf8b3a service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:28.042865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb66908-f8b4-4e3d-b91d-2b318abf8b3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:28.043331 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb66908-f8b4-4e3d-b91d-2b318abf8b3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:28.045339 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-cdb66908-f8b4-4e3d-b91d-2b318abf8b3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:28.101093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 976/1953] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:28.102378 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce19998-5138-41ca-a8ec-1efe0c13eb48 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:28.105125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4ce19998-5138-41ca-a8ec-1efe0c13eb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:28.123457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb66908-f8b4-4e3d-b91d-2b318abf8b3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:28.125878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce19998-5138-41ca-a8ec-1efe0c13eb48 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:28.127847 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb66908-f8b4-4e3d-b91d-2b318abf8b3a service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:28.136969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4cf58-234a-42eb-89c4-0fbfc9b6de1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:28.139947 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4cf58-234a-42eb-89c4-0fbfc9b6de1b service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:28.140420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4cf58-234a-42eb-89c4-0fbfc9b6de1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:28.140892 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4cf58-234a-42eb-89c4-0fbfc9b6de1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:28.146479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 978/1954] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:27 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:28.148040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff40ef7-01af-45d5-9ff4-f21443e6b254 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:28.148337 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff40ef7-01af-45d5-9ff4-f21443e6b254 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:28.148621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff40ef7-01af-45d5-9ff4-f21443e6b254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:28.150272 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:28.154677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4cf58-234a-42eb-89c4-0fbfc9b6de1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:28.165316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4cf58-234a-42eb-89c4-0fbfc9b6de1b service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:28.180962 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-6ff40ef7-01af-45d5-9ff4-f21443e6b254 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:28.184808 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-6ff40ef7-01af-45d5-9ff4-f21443e6b254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:28.189643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 977/1955] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:28.190617 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4cf58-234a-42eb-89c4-0fbfc9b6de1b service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:28.195237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f7d4cf58-234a-42eb-89c4-0fbfc9b6de1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:28.203888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 979/1956] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:28.204587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff40ef7-01af-45d5-9ff4-f21443e6b254 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:28.210458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-6ff40ef7-01af-45d5-9ff4-f21443e6b254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:28.231769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff40ef7-01af-45d5-9ff4-f21443e6b254 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:28.237023 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4cf58-234a-42eb-89c4-0fbfc9b6de1b service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:28.242701 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e74461-3a08-4632-9218-b4748f6fb8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:28.243726 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e74461-3a08-4632-9218-b4748f6fb8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:28.244068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e74461-3a08-4632-9218-b4748f6fb8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:28.244410 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e74461-3a08-4632-9218-b4748f6fb8e1 admin admin] PATH_INFO: `/v3/users/d25651f351294c6ab30291b0a04b149e` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:28.245981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e74461-3a08-4632-9218-b4748f6fb8e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d25651f351294c6ab30291b0a04b149e)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:28.247600 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a094d-81fa-476e-ab01-9c23e3a37237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:28.248219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e74461-3a08-4632-9218-b4748f6fb8e1 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:28.249473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a094d-81fa-476e-ab01-9c23e3a37237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:28.249473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a094d-81fa-476e-ab01-9c23e3a37237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:28.249473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a094d-81fa-476e-ab01-9c23e3a37237 admin admin] PATH_INFO: `/v3/projects/d11d62c5e61f4f07b4c061830348a595` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:28.250362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a094d-81fa-476e-ab01-9c23e3a37237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d11d62c5e61f4f07b4c061830348a595)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:28.252318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810a094d-81fa-476e-ab01-9c23e3a37237 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:28.254911 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:28.301505 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e6e74461-3a08-4632-9218-b4748f6fb8e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd25651f351294c6ab30291b0a04b149e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:28.302031 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-810a094d-81fa-476e-ab01-9c23e3a37237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd11d62c5e61f4f07b4c061830348a595', 'request_id': 'req-810a094d-81fa-476e-ab01-9c23e3a37237'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:28.308596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-810a094d-81fa-476e-ab01-9c23e3a37237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd11d62c5e61f4f07b4c061830348a595', 'request_id': 'req-810a094d-81fa-476e-ab01-9c23e3a37237'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:28.314446 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e6e74461-3a08-4632-9218-b4748f6fb8e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd25651f351294c6ab30291b0a04b149e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:28.314730 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-810a094d-81fa-476e-ab01-9c23e3a37237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd11d62c5e61f4f07b4c061830348a595', 'request_id': 'req-810a094d-81fa-476e-ab01-9c23e3a37237'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:28.321898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e6e74461-3a08-4632-9218-b4748f6fb8e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd25651f351294c6ab30291b0a04b149e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:28.322621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e6e74461-3a08-4632-9218-b4748f6fb8e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd25651f351294c6ab30291b0a04b149e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:28.323269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e6e74461-3a08-4632-9218-b4748f6fb8e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd25651f351294c6ab30291b0a04b149e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:28.328629 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-810a094d-81fa-476e-ab01-9c23e3a37237 admin admin] The token cache is being invalidate because project d11d62c5e61f4f07b4c061830348a595 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:28.328927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-810a094d-81fa-476e-ab01-9c23e3a37237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:28.329643 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-810a094d-81fa-476e-ab01-9c23e3a37237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:28.331454 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 980/1957] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:28 2022] DELETE /identity/v3/projects/d11d62c5e61f4f07b4c061830348a595 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:28.335266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a094d-81fa-476e-ab01-9c23e3a37237 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:28.336564 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-810a094d-81fa-476e-ab01-9c23e3a37237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:28.336729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 978/1958] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:28 2022] DELETE /identity/v3/users/d25651f351294c6ab30291b0a04b149e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:28.337753 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e74461-3a08-4632-9218-b4748f6fb8e1 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:28.339114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e6e74461-3a08-4632-9218-b4748f6fb8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:28.394470 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e74461-3a08-4632-9218-b4748f6fb8e1 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:28.398169 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c8874-1e2a-4d29-a02c-c108c830eeaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:28.398847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c8874-1e2a-4d29-a02c-c108c830eeaf service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:28.399146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c8874-1e2a-4d29-a02c-c108c830eeaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:28.399427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c8874-1e2a-4d29-a02c-c108c830eeaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:28.399794 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810a094d-81fa-476e-ab01-9c23e3a37237 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:28.401238 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1e3c8874-1e2a-4d29-a02c-c108c830eeaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:28.457205 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c55aa-d67c-4899-b09f-021c5cffbc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:28.457882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c55aa-d67c-4899-b09f-021c5cffbc17 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:28.458168 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c55aa-d67c-4899-b09f-021c5cffbc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:28.458441 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c55aa-d67c-4899-b09f-021c5cffbc17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:28.459212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c55aa-d67c-4899-b09f-021c5cffbc17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:44:28.459527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c55aa-d67c-4899-b09f-021c5cffbc17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:28.461198 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c55aa-d67c-4899-b09f-021c5cffbc17 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:28.461582 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c8874-1e2a-4d29-a02c-c108c830eeaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:28.463167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c8874-1e2a-4d29-a02c-c108c830eeaf service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:28.472512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 979/1959] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:28.473528 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c8874-1e2a-4d29-a02c-c108c830eeaf service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:28.474516 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 981/1960] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:44:28.474810 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1e3c8874-1e2a-4d29-a02c-c108c830eeaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:28.475721 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2502f2-f176-4f2f-b38e-8a91887b5f62 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:28.476009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2502f2-f176-4f2f-b38e-8a91887b5f62 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:28.476294 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2502f2-f176-4f2f-b38e-8a91887b5f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:28.489329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c8874-1e2a-4d29-a02c-c108c830eeaf service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:28.494251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5461c-4014-4ff8-86a3-9869b8a41aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:28.498603 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-3b2502f2-f176-4f2f-b38e-8a91887b5f62 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:28.498705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5461c-4014-4ff8-86a3-9869b8a41aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:28.498705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5461c-4014-4ff8-86a3-9869b8a41aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:28.498705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5461c-4014-4ff8-86a3-9869b8a41aeb admin admin] PATH_INFO: `/v3/users/58ed2f0c28334a0fb6e712678a60561c` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:28.498943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3b2502f2-f176-4f2f-b38e-8a91887b5f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:28.499050 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5461c-4014-4ff8-86a3-9869b8a41aeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=58ed2f0c28334a0fb6e712678a60561c)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:28.499330 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5461c-4014-4ff8-86a3-9869b8a41aeb admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:28.504966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 982/1961] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:28.509563 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2502f2-f176-4f2f-b38e-8a91887b5f62 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:28.520598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:28.521413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2502f2-f176-4f2f-b38e-8a91887b5f62 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:28.526673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfdca2-8eb1-47c2-bb6d-daafe062733a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:28.528016 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfdca2-8eb1-47c2-bb6d-daafe062733a admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:28.528390 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfdca2-8eb1-47c2-bb6d-daafe062733a admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:28.528756 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfdca2-8eb1-47c2-bb6d-daafe062733a admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:28.529673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfdca2-8eb1-47c2-bb6d-daafe062733a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:28.531642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfdca2-8eb1-47c2-bb6d-daafe062733a admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:28.561035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-2cd5461c-4014-4ff8-86a3-9869b8a41aeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58ed2f0c28334a0fb6e712678a60561c'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:28.565623 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 983/1962] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:28 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:28.566396 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfdca2-8eb1-47c2-bb6d-daafe062733a admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:28.568200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-65cfdca2-8eb1-47c2-bb6d-daafe062733a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:28.574328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-2cd5461c-4014-4ff8-86a3-9869b8a41aeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58ed2f0c28334a0fb6e712678a60561c'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:28.574923 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:28.584175 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-2cd5461c-4014-4ff8-86a3-9869b8a41aeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58ed2f0c28334a0fb6e712678a60561c'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:28.585292 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-2cd5461c-4014-4ff8-86a3-9869b8a41aeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58ed2f0c28334a0fb6e712678a60561c'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:28.585951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-2cd5461c-4014-4ff8-86a3-9869b8a41aeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58ed2f0c28334a0fb6e712678a60561c'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:28.600739 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 980/1963] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:28 2022] DELETE /identity/v3/users/58ed2f0c28334a0fb6e712678a60561c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:28.603874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5461c-4014-4ff8-86a3-9869b8a41aeb admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:28.603874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2cd5461c-4014-4ff8-86a3-9869b8a41aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:28.639094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfdca2-8eb1-47c2-bb6d-daafe062733a admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:28.713457 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5461c-4014-4ff8-86a3-9869b8a41aeb admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:28.718103 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21782430-cf5c-457c-9d45-7ebd292bca28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:28.719198 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21782430-cf5c-457c-9d45-7ebd292bca28 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:28.719533 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21782430-cf5c-457c-9d45-7ebd292bca28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:28.719843 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21782430-cf5c-457c-9d45-7ebd292bca28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:28.720570 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc2cf8-d9a9-474f-a856-d3b33fcc02d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:28.721558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc2cf8-d9a9-474f-a856-d3b33fcc02d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:28.721834 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-21782430-cf5c-457c-9d45-7ebd292bca28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:28.721982 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc2cf8-d9a9-474f-a856-d3b33fcc02d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:28.722158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc2cf8-d9a9-474f-a856-d3b33fcc02d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:28.723408 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc2cf8-d9a9-474f-a856-d3b33fcc02d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:28.726043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc2cf8-d9a9-474f-a856-d3b33fcc02d0 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:28.737701 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21782430-cf5c-457c-9d45-7ebd292bca28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:28.739979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21782430-cf5c-457c-9d45-7ebd292bca28 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:28.754316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 984/1964] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:28.756519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 981/1965] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:28 2022] POST /identity/v3/users => generated 283 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:28.761666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21782430-cf5c-457c-9d45-7ebd292bca28 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:28.773151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21782430-cf5c-457c-9d45-7ebd292bca28 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:28.778469 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef73801-ce24-4eb6-8247-f15fef3815a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:28.779221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef73801-ce24-4eb6-8247-f15fef3815a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:28.779511 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef73801-ce24-4eb6-8247-f15fef3815a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:28.779793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef73801-ce24-4eb6-8247-f15fef3815a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:28.780650 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef73801-ce24-4eb6-8247-f15fef3815a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:28.782279 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef73801-ce24-4eb6-8247-f15fef3815a0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:28.802896 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 985/1966] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:28.813482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc2cf8-d9a9-474f-a856-d3b33fcc02d0 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:28.842213 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc2cf8-d9a9-474f-a856-d3b33fcc02d0 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:28.849812 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb3e6d-18b1-4ed9-a2f3-18e9a2c34c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:28.850659 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb3e6d-18b1-4ed9-a2f3-18e9a2c34c1c admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:28.851030 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb3e6d-18b1-4ed9-a2f3-18e9a2c34c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:28.851445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb3e6d-18b1-4ed9-a2f3-18e9a2c34c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:28.852281 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb3e6d-18b1-4ed9-a2f3-18e9a2c34c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:28.854175 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb3e6d-18b1-4ed9-a2f3-18e9a2c34c1c admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:28.866467 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 982/1967] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:28.870989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef73801-ce24-4eb6-8247-f15fef3815a0 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:28.883686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef73801-ce24-4eb6-8247-f15fef3815a0 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:28.890613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ae8a3-2639-4485-af83-05aeaf39c09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:28.891343 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ae8a3-2639-4485-af83-05aeaf39c09f admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:28.891632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ae8a3-2639-4485-af83-05aeaf39c09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:28.891957 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ae8a3-2639-4485-af83-05aeaf39c09f admin admin] PATH_INFO: `/v3/projects/7949453e518b4f11818d4b43a6857321/users/b7ed8a1b6122461ab40095df98473d6c/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:28.911125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ae8a3-2639-4485-af83-05aeaf39c09f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7949453e518b4f11818d4b43a6857321, user_id=b7ed8a1b6122461ab40095df98473d6c, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:28.914791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ae8a3-2639-4485-af83-05aeaf39c09f admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:28.940092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 986/1968] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:28 2022] PUT /identity/v3/projects/7949453e518b4f11818d4b43a6857321/users/b7ed8a1b6122461ab40095df98473d6c/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:28.952719 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fc64c-bfec-4f59-aa90-8358786fe3a5 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:28.953013 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fc64c-bfec-4f59-aa90-8358786fe3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:28.953306 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fc64c-bfec-4f59-aa90-8358786fe3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:28.988017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-7b6fc64c-bfec-4f59-aa90-8358786fe3a5 None None] MFA Rules not processed for user `b7ed8a1b6122461ab40095df98473d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:28.989337 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-7b6fc64c-bfec-4f59-aa90-8358786fe3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:29.083131 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 983/1969] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:29.095655 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ae8a3-2639-4485-af83-05aeaf39c09f admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:29.097456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ad8ae8a3-2639-4485-af83-05aeaf39c09f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:29.125017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fc64c-bfec-4f59-aa90-8358786fe3a5 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:29.126472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-7b6fc64c-bfec-4f59-aa90-8358786fe3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:29.132724 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:29.257556 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ae8a3-2639-4485-af83-05aeaf39c09f admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:29.263535 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6bcc9-dee1-4495-bd87-343915eb54dd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2707c6967a0a419a98ae142ff1d1a961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:29.264767 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6bcc9-dee1-4495-bd87-343915eb54dd service glance] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:29.265341 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6bcc9-dee1-4495-bd87-343915eb54dd service glance] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:29.265947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6bcc9-dee1-4495-bd87-343915eb54dd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:29.268309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6fc64c-bfec-4f59-aa90-8358786fe3a5 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:29.273368 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45bd4a-9e01-4fac-9bd5-132fe2700529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:29.274275 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45bd4a-9e01-4fac-9bd5-132fe2700529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:29.274609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45bd4a-9e01-4fac-9bd5-132fe2700529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:29.275605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45bd4a-9e01-4fac-9bd5-132fe2700529 admin admin] PATH_INFO: `/v3/projects/4cb3beee5c5244ba83ed6c0edaca59b9` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:29.276283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45bd4a-9e01-4fac-9bd5-132fe2700529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cb3beee5c5244ba83ed6c0edaca59b9)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:29.278252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45bd4a-9e01-4fac-9bd5-132fe2700529 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:29.281017 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d6bcc9-dee1-4495-bd87-343915eb54dd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:29.281017 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d6bcc9-dee1-4495-bd87-343915eb54dd service glance] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:29.292472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 987/1970] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:29.293433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6bcc9-dee1-4495-bd87-343915eb54dd service glance] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:29.294860 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b6d6bcc9-dee1-4495-bd87-343915eb54dd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:29.313889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6bcc9-dee1-4495-bd87-343915eb54dd service glance] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:29.320166 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816cc22-b742-4092-9e4e-a8752ffe34f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:29.321167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816cc22-b742-4092-9e4e-a8752ffe34f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:29.321520 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816cc22-b742-4092-9e4e-a8752ffe34f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:29.321875 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816cc22-b742-4092-9e4e-a8752ffe34f2 admin admin] PATH_INFO: `/v3/projects/3b7d57b0794c48e2826302b1c58232bf` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:29.323229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816cc22-b742-4092-9e4e-a8752ffe34f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b7d57b0794c48e2826302b1c58232bf)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:29.328186 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816cc22-b742-4092-9e4e-a8752ffe34f2 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:29.335003 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:29.352028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6f45bd4a-9e01-4fac-9bd5-132fe2700529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cb3beee5c5244ba83ed6c0edaca59b9', 'request_id': 'req-6f45bd4a-9e01-4fac-9bd5-132fe2700529'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:29.362868 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6f45bd4a-9e01-4fac-9bd5-132fe2700529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cb3beee5c5244ba83ed6c0edaca59b9', 'request_id': 'req-6f45bd4a-9e01-4fac-9bd5-132fe2700529'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:29.379766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6f45bd4a-9e01-4fac-9bd5-132fe2700529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cb3beee5c5244ba83ed6c0edaca59b9', 'request_id': 'req-6f45bd4a-9e01-4fac-9bd5-132fe2700529'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:29.389309 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-a816cc22-b742-4092-9e4e-a8752ffe34f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b7d57b0794c48e2826302b1c58232bf', 'request_id': 'req-a816cc22-b742-4092-9e4e-a8752ffe34f2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:29.396597 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6f45bd4a-9e01-4fac-9bd5-132fe2700529 admin admin] The token cache is being invalidate because project 4cb3beee5c5244ba83ed6c0edaca59b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:29.397725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6f45bd4a-9e01-4fac-9bd5-132fe2700529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:29.398224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-a816cc22-b742-4092-9e4e-a8752ffe34f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b7d57b0794c48e2826302b1c58232bf', 'request_id': 'req-a816cc22-b742-4092-9e4e-a8752ffe34f2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:29.398995 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6f45bd4a-9e01-4fac-9bd5-132fe2700529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:29.401876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 984/1971] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:29 2022] DELETE /identity/v3/projects/4cb3beee5c5244ba83ed6c0edaca59b9 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:29.406209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-a816cc22-b742-4092-9e4e-a8752ffe34f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b7d57b0794c48e2826302b1c58232bf', 'request_id': 'req-a816cc22-b742-4092-9e4e-a8752ffe34f2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:29.409095 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39734bc-60f6-4d21-a0c7-fee2429a098f None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:29.410452 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39734bc-60f6-4d21-a0c7-fee2429a098f None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:29.410792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39734bc-60f6-4d21-a0c7-fee2429a098f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:29.432257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-a816cc22-b742-4092-9e4e-a8752ffe34f2 admin admin] The token cache is being invalidate because project 3b7d57b0794c48e2826302b1c58232bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:29.432640 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-a816cc22-b742-4092-9e4e-a8752ffe34f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:29.433375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-a816cc22-b742-4092-9e4e-a8752ffe34f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:29.434253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-b39734bc-60f6-4d21-a0c7-fee2429a098f None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:29.435452 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 988/1972] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:29 2022] DELETE /identity/v3/projects/3b7d57b0794c48e2826302b1c58232bf => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:29.435696 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b39734bc-60f6-4d21-a0c7-fee2429a098f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:29.440679 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29515f-8539-4b89-bdc5-8486961cb4b9 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:29.441201 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29515f-8539-4b89-bdc5-8486961cb4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:29.441491 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29515f-8539-4b89-bdc5-8486961cb4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:29.498891 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-af29515f-8539-4b89-bdc5-8486961cb4b9 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:29.500646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-af29515f-8539-4b89-bdc5-8486961cb4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:29.519991 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 985/1973] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:44:29.521434 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f350dc-7554-47fb-8bac-5f63d65cccb4 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:29.521864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f350dc-7554-47fb-8bac-5f63d65cccb4 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:29.522152 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f350dc-7554-47fb-8bac-5f63d65cccb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:29.523630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 989/1974] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:29.525154 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15f951-eb71-4010-bdaa-58e0d6e1bac9 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:29.525459 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15f951-eb71-4010-bdaa-58e0d6e1bac9 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:29.525744 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15f951-eb71-4010-bdaa-58e0d6e1bac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:29.546845 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-9a15f951-eb71-4010-bdaa-58e0d6e1bac9 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:29.547461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-26f350dc-7554-47fb-8bac-5f63d65cccb4 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:29.548104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9a15f951-eb71-4010-bdaa-58e0d6e1bac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:29.548936 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-26f350dc-7554-47fb-8bac-5f63d65cccb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:29.557604 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 990/1975] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:29.561221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 986/1976] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:29.563195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f350dc-7554-47fb-8bac-5f63d65cccb4 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:29.563311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15f951-eb71-4010-bdaa-58e0d6e1bac9 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:29.573876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f350dc-7554-47fb-8bac-5f63d65cccb4 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:29.575501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15f951-eb71-4010-bdaa-58e0d6e1bac9 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:29.580906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d946481-c886-401c-9c79-94a79a4a0f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:29.582989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d946481-c886-401c-9c79-94a79a4a0f2a admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:29.583832 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d946481-c886-401c-9c79-94a79a4a0f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:29.584324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d946481-c886-401c-9c79-94a79a4a0f2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:29.584804 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e36072-75d9-4cdc-aa99-b1627699903f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:29.585497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e36072-75d9-4cdc-aa99-b1627699903f admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:29.585873 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d946481-c886-401c-9c79-94a79a4a0f2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:44:29.585980 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e36072-75d9-4cdc-aa99-b1627699903f admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:29.586227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e36072-75d9-4cdc-aa99-b1627699903f admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:29.586712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d946481-c886-401c-9c79-94a79a4a0f2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:29.587574 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e36072-75d9-4cdc-aa99-b1627699903f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:44:29.587789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e36072-75d9-4cdc-aa99-b1627699903f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:29.589622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d946481-c886-401c-9c79-94a79a4a0f2a admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:29.594510 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e36072-75d9-4cdc-aa99-b1627699903f admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:29.607549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 991/1977] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:44:29.608734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d946481-c886-401c-9c79-94a79a4a0f2a admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:29.612153 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 987/1978] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:44:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:44:29.620401 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e36072-75d9-4cdc-aa99-b1627699903f admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:29.629222 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d946481-c886-401c-9c79-94a79a4a0f2a admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:29.632912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e36072-75d9-4cdc-aa99-b1627699903f admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:29.638037 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344b486-64af-4302-95bf-7d4236df3b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:29.638852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522c370-5a4e-4e9d-969c-a9ad99fcf845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:29.639616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522c370-5a4e-4e9d-969c-a9ad99fcf845 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:29.639984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522c370-5a4e-4e9d-969c-a9ad99fcf845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:29.640724 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522c370-5a4e-4e9d-969c-a9ad99fcf845 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:29.640839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344b486-64af-4302-95bf-7d4236df3b5b admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:29.641244 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344b486-64af-4302-95bf-7d4236df3b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:29.641420 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344b486-64af-4302-95bf-7d4236df3b5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:29.641577 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522c370-5a4e-4e9d-969c-a9ad99fcf845 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:29.642803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344b486-64af-4302-95bf-7d4236df3b5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:29.643536 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522c370-5a4e-4e9d-969c-a9ad99fcf845 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:29.644890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344b486-64af-4302-95bf-7d4236df3b5b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:29.668559 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:29.700224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 988/1979] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:29 2022] POST /identity/v3/projects => generated 375 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:29.703200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 992/1980] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:29 2022] POST /identity/v3/projects => generated 399 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:29.705190 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522c370-5a4e-4e9d-969c-a9ad99fcf845 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:29.708093 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344b486-64af-4302-95bf-7d4236df3b5b admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:29.782330 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344b486-64af-4302-95bf-7d4236df3b5b admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:29.802839 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522c370-5a4e-4e9d-969c-a9ad99fcf845 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:29.810700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ca4dc-d58e-48ea-b2aa-ffdccca2050a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:29.811910 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ca4dc-d58e-48ea-b2aa-ffdccca2050a admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:29.812206 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ca4dc-d58e-48ea-b2aa-ffdccca2050a admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:29.812454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ca4dc-d58e-48ea-b2aa-ffdccca2050a admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:29.813170 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ca4dc-d58e-48ea-b2aa-ffdccca2050a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:29.815895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ca4dc-d58e-48ea-b2aa-ffdccca2050a admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:29.846879 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 989/1981] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:29 2022] POST /identity/v3/users => generated 300 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:29.852621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ca4dc-d58e-48ea-b2aa-ffdccca2050a admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:29.864399 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ca4dc-d58e-48ea-b2aa-ffdccca2050a admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:29.871802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46a6c4-5632-4134-be98-d17bdc36e9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:29.872893 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46a6c4-5632-4134-be98-d17bdc36e9c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:29.873233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46a6c4-5632-4134-be98-d17bdc36e9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:29.873537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46a6c4-5632-4134-be98-d17bdc36e9c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:29.874283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46a6c4-5632-4134-be98-d17bdc36e9c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:29.875706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:29.875938 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46a6c4-5632-4134-be98-d17bdc36e9c9 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:29.889367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 990/1982] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:44:29.893127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c5544-33e8-448a-b90d-4ae98677737a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:29.893979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46a6c4-5632-4134-be98-d17bdc36e9c9 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:29.894094 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c5544-33e8-448a-b90d-4ae98677737a admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:29.894260 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c5544-33e8-448a-b90d-4ae98677737a admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:29.894581 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c5544-33e8-448a-b90d-4ae98677737a admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:29.895324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c5544-33e8-448a-b90d-4ae98677737a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:29.897361 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c5544-33e8-448a-b90d-4ae98677737a admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:29.907834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46a6c4-5632-4134-be98-d17bdc36e9c9 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:29.928061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 993/1983] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:29 2022] POST /identity/v3/users => generated 288 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:29.928528 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce189498-56f9-4eb5-8572-4a60aa365e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:29.929518 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce189498-56f9-4eb5-8572-4a60aa365e4e admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:29.930005 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce189498-56f9-4eb5-8572-4a60aa365e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:29.930554 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce189498-56f9-4eb5-8572-4a60aa365e4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:29.931604 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce189498-56f9-4eb5-8572-4a60aa365e4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:29.934183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c5544-33e8-448a-b90d-4ae98677737a admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:29.941947 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce189498-56f9-4eb5-8572-4a60aa365e4e admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:29.953288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c5544-33e8-448a-b90d-4ae98677737a admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:29.960105 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe8641-8edf-4814-8fb4-e974abf9155d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:29.961096 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe8641-8edf-4814-8fb4-e974abf9155d admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:29.961425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe8641-8edf-4814-8fb4-e974abf9155d admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:29.961697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe8641-8edf-4814-8fb4-e974abf9155d admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:29.962565 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfe8641-8edf-4814-8fb4-e974abf9155d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:29.964443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfe8641-8edf-4814-8fb4-e974abf9155d admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:29.972260 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 991/1984] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:29.976611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce189498-56f9-4eb5-8572-4a60aa365e4e admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:29.983065 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 994/1985] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:29.987996 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe8641-8edf-4814-8fb4-e974abf9155d admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:29.999117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce189498-56f9-4eb5-8572-4a60aa365e4e admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:30.018373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe8641-8edf-4814-8fb4-e974abf9155d admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:30.024059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9b0eb-47b6-44ea-9829-ba3db51a3b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:30.024959 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9b0eb-47b6-44ea-9829-ba3db51a3b85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.025241 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9b0eb-47b6-44ea-9829-ba3db51a3b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.025513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9b0eb-47b6-44ea-9829-ba3db51a3b85 admin admin] PATH_INFO: `/v3/projects/4ff44cd9d5c04fb5bf58db66d2a7e16b/users/f21724b0629a4ba59583ac9cee837506/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.028619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f4f5c-e310-451d-b35e-05775f269431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:30.030029 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f4f5c-e310-451d-b35e-05775f269431 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.030524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f4f5c-e310-451d-b35e-05775f269431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.031042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f4f5c-e310-451d-b35e-05775f269431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.032249 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f4f5c-e310-451d-b35e-05775f269431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:30.034248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f4f5c-e310-451d-b35e-05775f269431 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:30.045659 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9b0eb-47b6-44ea-9829-ba3db51a3b85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff44cd9d5c04fb5bf58db66d2a7e16b, user_id=f21724b0629a4ba59583ac9cee837506, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:30.050109 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9b0eb-47b6-44ea-9829-ba3db51a3b85 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:30.053326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 995/1986] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:30.058006 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f4f5c-e310-451d-b35e-05775f269431 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:30.069108 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 992/1987] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:29 2022] PUT /identity/v3/projects/4ff44cd9d5c04fb5bf58db66d2a7e16b/users/f21724b0629a4ba59583ac9cee837506/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:30.071458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f4f5c-e310-451d-b35e-05775f269431 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:30.080761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfac6c-9df5-4d3d-822c-19bc456c5f43 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.081213 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfac6c-9df5-4d3d-822c-19bc456c5f43 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.082006 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfac6c-9df5-4d3d-822c-19bc456c5f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.114363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-b9cfac6c-9df5-4d3d-822c-19bc456c5f43 None None] MFA Rules not processed for user `f21724b0629a4ba59583ac9cee837506`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:30.115665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b9cfac6c-9df5-4d3d-822c-19bc456c5f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:30.162844 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9faeb07-1b35-40f1-90a4-cac3364d1dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:30.163882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9faeb07-1b35-40f1-90a4-cac3364d1dfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.164163 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9faeb07-1b35-40f1-90a4-cac3364d1dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.164437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9faeb07-1b35-40f1-90a4-cac3364d1dfb admin admin] PATH_INFO: `/v3/projects/fe1071f281434e18877a75fc8b6d6725/users/10c3fa2bbc844cbfbf484d9fc66bedbd/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.189393 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9faeb07-1b35-40f1-90a4-cac3364d1dfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe1071f281434e18877a75fc8b6d6725, user_id=10c3fa2bbc844cbfbf484d9fc66bedbd, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:30.192303 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9faeb07-1b35-40f1-90a4-cac3364d1dfb admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:30.198775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 993/1988] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:44:30.200196 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb27d7-17c5-479f-8338-64fd6444e83c None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.200585 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb27d7-17c5-479f-8338-64fd6444e83c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.200920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb27d7-17c5-479f-8338-64fd6444e83c None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.202898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 994/1989] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jul 21 05:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:44:30.203813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb27d7-17c5-479f-8338-64fd6444e83c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:30.211459 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 996/1990] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:30 2022] PUT /identity/v3/projects/fe1071f281434e18877a75fc8b6d6725/users/10c3fa2bbc844cbfbf484d9fc66bedbd/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:30.213011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312f62e-aaa6-4576-bb9d-cae2634dca33 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.213011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312f62e-aaa6-4576-bb9d-cae2634dca33 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.213314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312f62e-aaa6-4576-bb9d-cae2634dca33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.236348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-e312f62e-aaa6-4576-bb9d-cae2634dca33 None None] MFA Rules not processed for user `928d014648cd41d7b58e7e01845eb748`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:30.237574 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-e312f62e-aaa6-4576-bb9d-cae2634dca33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:30.245951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:30.288594 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fb27d7-17c5-479f-8338-64fd6444e83c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:30.293585 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f54ac-2973-430a-ad38-03df82955d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:30.294264 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f54ac-2973-430a-ad38-03df82955d6b admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.294639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f54ac-2973-430a-ad38-03df82955d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.294927 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f54ac-2973-430a-ad38-03df82955d6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.295732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f54ac-2973-430a-ad38-03df82955d6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:30.297720 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f54ac-2973-430a-ad38-03df82955d6b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:30.319704 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 997/1991] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jul 21 05:44:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:30.321167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728683f5-3844-4169-9732-384c9e89427c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.321459 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728683f5-3844-4169-9732-384c9e89427c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.321734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728683f5-3844-4169-9732-384c9e89427c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.329560 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 995/1992] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:30 2022] POST /identity/v3/projects => generated 399 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:30.332358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f54ac-2973-430a-ad38-03df82955d6b admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:30.334271 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-722f54ac-2973-430a-ad38-03df82955d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:30.349351 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-728683f5-3844-4169-9732-384c9e89427c None None] MFA Rules not processed for user `10c3fa2bbc844cbfbf484d9fc66bedbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:30.350560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-728683f5-3844-4169-9732-384c9e89427c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:30.376454 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:30.416901 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f54ac-2973-430a-ad38-03df82955d6b admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:30.419765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 998/1993] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:30.419923 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728683f5-3844-4169-9732-384c9e89427c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:30.423707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71088c7e-f74f-4d87-a534-1a6a5e04e9f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:30.424526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71088c7e-f74f-4d87-a534-1a6a5e04e9f8 service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.424813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71088c7e-f74f-4d87-a534-1a6a5e04e9f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.425095 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71088c7e-f74f-4d87-a534-1a6a5e04e9f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.435871 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71088c7e-f74f-4d87-a534-1a6a5e04e9f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:30.437475 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71088c7e-f74f-4d87-a534-1a6a5e04e9f8 service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:30.448416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 996/1994] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:44:30.449264 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71088c7e-f74f-4d87-a534-1a6a5e04e9f8 service nova] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:30.494328 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728683f5-3844-4169-9732-384c9e89427c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:30.499357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bba0c-794b-4db2-b492-2ccfc6422eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:30.500521 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bba0c-794b-4db2-b492-2ccfc6422eed admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.500521 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bba0c-794b-4db2-b492-2ccfc6422eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.500698 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bba0c-794b-4db2-b492-2ccfc6422eed admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.501326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854bba0c-794b-4db2-b492-2ccfc6422eed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:30.503347 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854bba0c-794b-4db2-b492-2ccfc6422eed admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:30.535380 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 999/1995] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:30 2022] POST /identity/v3/users => generated 306 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:30.539170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bba0c-794b-4db2-b492-2ccfc6422eed admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:30.548879 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71088c7e-f74f-4d87-a534-1a6a5e04e9f8 service nova] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:30.553972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bba0c-794b-4db2-b492-2ccfc6422eed admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:30.561455 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bc41c-4986-4047-85cf-58e397f94747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:30.562136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bc41c-4986-4047-85cf-58e397f94747 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.562429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bc41c-4986-4047-85cf-58e397f94747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.562688 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bc41c-4986-4047-85cf-58e397f94747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.564718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3bc41c-4986-4047-85cf-58e397f94747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:30.566471 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3bc41c-4986-4047-85cf-58e397f94747 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:30.567897 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063c735-90a6-46ab-9567-0853ffc614e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:30.569274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063c735-90a6-46ab-9567-0853ffc614e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.569274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063c735-90a6-46ab-9567-0853ffc614e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.569274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063c735-90a6-46ab-9567-0853ffc614e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.569777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1063c735-90a6-46ab-9567-0853ffc614e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:30.571730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1063c735-90a6-46ab-9567-0853ffc614e7 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:30.583386 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1000/1996] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:30.587719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bc41c-4986-4047-85cf-58e397f94747 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:30.599996 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bc41c-4986-4047-85cf-58e397f94747 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:30.606369 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e3d0a-069b-473d-bd63-2f8bd1d226fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:30.610228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e3d0a-069b-473d-bd63-2f8bd1d226fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.610228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e3d0a-069b-473d-bd63-2f8bd1d226fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.610228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e3d0a-069b-473d-bd63-2f8bd1d226fb admin admin] PATH_INFO: `/v3/projects/486153bad1504f1fa4190758e8a66668/users/4462b522f2614c83a8e42a7b2dc7a2e8/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.611060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 997/1997] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:30 2022] POST /identity/v3/projects => generated 373 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:30.612370 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063c735-90a6-46ab-9567-0853ffc614e7 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:30.628387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e3d0a-069b-473d-bd63-2f8bd1d226fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=486153bad1504f1fa4190758e8a66668, user_id=4462b522f2614c83a8e42a7b2dc7a2e8, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:30.631935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e3d0a-069b-473d-bd63-2f8bd1d226fb admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:30.641536 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:30.664911 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1001/1998] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:30 2022] PUT /identity/v3/projects/486153bad1504f1fa4190758e8a66668/users/4462b522f2614c83a8e42a7b2dc7a2e8/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:30.669154 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e3d0a-069b-473d-bd63-2f8bd1d226fb admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:30.691802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063c735-90a6-46ab-9567-0853ffc614e7 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:30.750906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e3d0a-069b-473d-bd63-2f8bd1d226fb admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:30.760362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068e06d-21e1-4f6b-b460-6d3212dfc8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:30.761064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068e06d-21e1-4f6b-b460-6d3212dfc8b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.761346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068e06d-21e1-4f6b-b460-6d3212dfc8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.761516 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d772a-deeb-413b-890a-c7407af8e991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:30.761635 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068e06d-21e1-4f6b-b460-6d3212dfc8b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.762381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d772a-deeb-413b-890a-c7407af8e991 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.767191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068e06d-21e1-4f6b-b460-6d3212dfc8b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:30.767408 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d772a-deeb-413b-890a-c7407af8e991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.767408 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d772a-deeb-413b-890a-c7407af8e991 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.767408 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d772a-deeb-413b-890a-c7407af8e991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:30.767650 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068e06d-21e1-4f6b-b460-6d3212dfc8b3 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:30.767738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d772a-deeb-413b-890a-c7407af8e991 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:30.789973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1002/2000] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:30.790981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068e06d-21e1-4f6b-b460-6d3212dfc8b3 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:30.792281 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 998/1999] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:30 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:30.793218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d772a-deeb-413b-890a-c7407af8e991 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:30.802460 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068e06d-21e1-4f6b-b460-6d3212dfc8b3 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:30.805151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d772a-deeb-413b-890a-c7407af8e991 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:30.807343 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a82eb43-fc79-4504-9802-20def3587bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:30.807868 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947f32b-5f03-4b3c-85b9-eac379f1d4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:30.808370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82eb43-fc79-4504-9802-20def3587bbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.808522 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947f32b-5f03-4b3c-85b9-eac379f1d4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.808764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82eb43-fc79-4504-9802-20def3587bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.808865 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947f32b-5f03-4b3c-85b9-eac379f1d4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.809066 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947f32b-5f03-4b3c-85b9-eac379f1d4a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.809261 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a82eb43-fc79-4504-9802-20def3587bbf admin admin] PATH_INFO: `/v3/projects/486153bad1504f1fa4190758e8a66668/users/4462b522f2614c83a8e42a7b2dc7a2e8/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.809797 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947f32b-5f03-4b3c-85b9-eac379f1d4a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:30.811587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a82eb43-fc79-4504-9802-20def3587bbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=486153bad1504f1fa4190758e8a66668, user_id=4462b522f2614c83a8e42a7b2dc7a2e8, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:30.811696 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947f32b-5f03-4b3c-85b9-eac379f1d4a7 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:30.815591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a82eb43-fc79-4504-9802-20def3587bbf admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:30.822696 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 999/2001] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:30.826867 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947f32b-5f03-4b3c-85b9-eac379f1d4a7 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:30.832517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1003/2002] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:30 2022] PUT /identity/v3/projects/486153bad1504f1fa4190758e8a66668/users/4462b522f2614c83a8e42a7b2dc7a2e8/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:30.836827 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947f32b-5f03-4b3c-85b9-eac379f1d4a7 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:30.842790 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5db44d-129a-4a70-8735-38d708fb33fc None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.843161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5db44d-129a-4a70-8735-38d708fb33fc None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.843457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5db44d-129a-4a70-8735-38d708fb33fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.852529 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:30.880599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-7d5db44d-129a-4a70-8735-38d708fb33fc None None] MFA Rules not processed for user `4462b522f2614c83a8e42a7b2dc7a2e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:30.882235 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7d5db44d-129a-4a70-8735-38d708fb33fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:30.905830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018ff9e-3062-40cd-aa80-b7a91b4a3d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:30.906811 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018ff9e-3062-40cd-aa80-b7a91b4a3d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:30.907178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018ff9e-3062-40cd-aa80-b7a91b4a3d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:30.907520 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018ff9e-3062-40cd-aa80-b7a91b4a3d48 admin admin] PATH_INFO: `/v3/projects/b65615acf98642679d6b605e205aab04/users/ca58209b7ca4483895a0afa5560dc9d5/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:30.931035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018ff9e-3062-40cd-aa80-b7a91b4a3d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b65615acf98642679d6b605e205aab04, user_id=ca58209b7ca4483895a0afa5560dc9d5, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:30.933779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018ff9e-3062-40cd-aa80-b7a91b4a3d48 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:30.948348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1000/2003] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:30 2022] PUT /identity/v3/projects/b65615acf98642679d6b605e205aab04/users/ca58209b7ca4483895a0afa5560dc9d5/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:30.952497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018ff9e-3062-40cd-aa80-b7a91b4a3d48 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:30.963822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1004/2004] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:30 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:30.982099 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5db44d-129a-4a70-8735-38d708fb33fc None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:30.983819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7d5db44d-129a-4a70-8735-38d708fb33fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:30.985651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:31.036293 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018ff9e-3062-40cd-aa80-b7a91b4a3d48 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:31.041361 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c554940-c70e-4cd2-a0ce-23f2b10c300c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:31.041990 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c554940-c70e-4cd2-a0ce-23f2b10c300c admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:31.042282 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c554940-c70e-4cd2-a0ce-23f2b10c300c admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:31.042563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c554940-c70e-4cd2-a0ce-23f2b10c300c admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:31.043432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c554940-c70e-4cd2-a0ce-23f2b10c300c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:31.045121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c554940-c70e-4cd2-a0ce-23f2b10c300c admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:31.055393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1001/2005] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:30 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:31.060419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c554940-c70e-4cd2-a0ce-23f2b10c300c admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:31.069449 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5db44d-129a-4a70-8735-38d708fb33fc None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:31.071957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c554940-c70e-4cd2-a0ce-23f2b10c300c admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:31.076986 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf073dfe-7f7b-4891-adb6-f72e51c84b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:31.077889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf073dfe-7f7b-4891-adb6-f72e51c84b39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:31.078178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf073dfe-7f7b-4891-adb6-f72e51c84b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:31.078461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf073dfe-7f7b-4891-adb6-f72e51c84b39 admin admin] PATH_INFO: `/v3/projects/b65615acf98642679d6b605e205aab04/users/ca58209b7ca4483895a0afa5560dc9d5/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:31.080484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf073dfe-7f7b-4891-adb6-f72e51c84b39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b65615acf98642679d6b605e205aab04, user_id=ca58209b7ca4483895a0afa5560dc9d5, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:31.083285 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf073dfe-7f7b-4891-adb6-f72e51c84b39 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:31.095343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1002/2006] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:31 2022] PUT /identity/v3/projects/b65615acf98642679d6b605e205aab04/users/ca58209b7ca4483895a0afa5560dc9d5/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:31.106009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57035e94-4f27-4f68-b6d8-c83ceea576c0 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:31.106306 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57035e94-4f27-4f68-b6d8-c83ceea576c0 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:31.106585 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57035e94-4f27-4f68-b6d8-c83ceea576c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:31.135804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43382f9-b3fb-43f0-a42f-186b67c21392 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:31.136495 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43382f9-b3fb-43f0-a42f-186b67c21392 service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:31.136763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43382f9-b3fb-43f0-a42f-186b67c21392 service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:31.137043 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43382f9-b3fb-43f0-a42f-186b67c21392 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:31.142334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-57035e94-4f27-4f68-b6d8-c83ceea576c0 None None] MFA Rules not processed for user `ca58209b7ca4483895a0afa5560dc9d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:31.144122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-57035e94-4f27-4f68-b6d8-c83ceea576c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:31.205373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:31.208762 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43382f9-b3fb-43f0-a42f-186b67c21392 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:31.210378 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43382f9-b3fb-43f0-a42f-186b67c21392 service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:31.221269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1005/2007] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:30 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:31.222734 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1003/2008] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:31.234301 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43382f9-b3fb-43f0-a42f-186b67c21392 service cinder] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:31.235799 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a43382f9-b3fb-43f0-a42f-186b67c21392 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:31.247289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43382f9-b3fb-43f0-a42f-186b67c21392 service cinder] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:31.305006 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6a613-48fa-4f59-a853-71a15c448fbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:31.305799 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6a613-48fa-4f59-a853-71a15c448fbc service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:31.306091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6a613-48fa-4f59-a853-71a15c448fbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:31.306416 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6a613-48fa-4f59-a853-71a15c448fbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:31.367321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57035e94-4f27-4f68-b6d8-c83ceea576c0 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:31.373055 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6a613-48fa-4f59-a853-71a15c448fbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:31.374528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6a613-48fa-4f59-a853-71a15c448fbc service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:31.377733 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57035e94-4f27-4f68-b6d8-c83ceea576c0 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:31.381894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec17da-20c9-48d1-bbc1-7970f4afcf4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:31.382548 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec17da-20c9-48d1-bbc1-7970f4afcf4e service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:31.382837 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec17da-20c9-48d1-bbc1-7970f4afcf4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:31.383230 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec17da-20c9-48d1-bbc1-7970f4afcf4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:31.384282 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1006/2009] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:31.445438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec17da-20c9-48d1-bbc1-7970f4afcf4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:31.446935 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec17da-20c9-48d1-bbc1-7970f4afcf4e service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:31.455986 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1004/2010] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:31 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:37.173769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6a613-48fa-4f59-a853-71a15c448fbc service cinder] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:37.308422 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6a613-48fa-4f59-a853-71a15c448fbc service cinder] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:37.314301 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bed1aa-8657-49d0-8d87-d3200108f6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:37.315383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bed1aa-8657-49d0-8d87-d3200108f6a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:37.315997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bed1aa-8657-49d0-8d87-d3200108f6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:37.315997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bed1aa-8657-49d0-8d87-d3200108f6a0 admin admin] PATH_INFO: `/v3/users/10c3fa2bbc844cbfbf484d9fc66bedbd` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:37.317228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bed1aa-8657-49d0-8d87-d3200108f6a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10c3fa2bbc844cbfbf484d9fc66bedbd)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:37.319361 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bed1aa-8657-49d0-8d87-d3200108f6a0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:37.330754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec17da-20c9-48d1-bbc1-7970f4afcf4e service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:37.332900 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b8ec17da-20c9-48d1-bbc1-7970f4afcf4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:37.351344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec17da-20c9-48d1-bbc1-7970f4afcf4e service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:37.357790 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612071e0-3a7e-4fab-8cf4-9052e92e905e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:37.359110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612071e0-3a7e-4fab-8cf4-9052e92e905e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:37.364942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612071e0-3a7e-4fab-8cf4-9052e92e905e admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:37.364942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612071e0-3a7e-4fab-8cf4-9052e92e905e admin admin] PATH_INFO: `/v3/users/392acb39b08f4c1dbb30289fcab43990` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:37.364942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612071e0-3a7e-4fab-8cf4-9052e92e905e admin admin] RBAC: Authorizing `identity:delete_user(user_id=392acb39b08f4c1dbb30289fcab43990)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:37.364942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612071e0-3a7e-4fab-8cf4-9052e92e905e admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:37.451780 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f7bed1aa-8657-49d0-8d87-d3200108f6a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10c3fa2bbc844cbfbf484d9fc66bedbd'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:37.454705 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:37.457285 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:37.476362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f7bed1aa-8657-49d0-8d87-d3200108f6a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10c3fa2bbc844cbfbf484d9fc66bedbd'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:37.487684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-612071e0-3a7e-4fab-8cf4-9052e92e905e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '392acb39b08f4c1dbb30289fcab43990'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:37.492635 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f7bed1aa-8657-49d0-8d87-d3200108f6a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10c3fa2bbc844cbfbf484d9fc66bedbd'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:37.496372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f7bed1aa-8657-49d0-8d87-d3200108f6a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10c3fa2bbc844cbfbf484d9fc66bedbd'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:37.498539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f7bed1aa-8657-49d0-8d87-d3200108f6a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10c3fa2bbc844cbfbf484d9fc66bedbd'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:37.523178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-612071e0-3a7e-4fab-8cf4-9052e92e905e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '392acb39b08f4c1dbb30289fcab43990'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:37.537085 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-612071e0-3a7e-4fab-8cf4-9052e92e905e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '392acb39b08f4c1dbb30289fcab43990'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:37.537828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-612071e0-3a7e-4fab-8cf4-9052e92e905e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '392acb39b08f4c1dbb30289fcab43990'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:37.538602 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-612071e0-3a7e-4fab-8cf4-9052e92e905e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '392acb39b08f4c1dbb30289fcab43990'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:37.544101 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1007/2011] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:37 2022] DELETE /identity/v3/users/10c3fa2bbc844cbfbf484d9fc66bedbd => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:37.546159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e239cff-2b53-4495-9a39-13b98f351363 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:37.546636 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e239cff-2b53-4495-9a39-13b98f351363 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:37.547125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e239cff-2b53-4495-9a39-13b98f351363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:37.571474 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1005/2012] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:37 2022] DELETE /identity/v3/users/392acb39b08f4c1dbb30289fcab43990 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:37.576351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06170128-8f26-43c1-934f-3008c23fbb95 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:37.576858 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06170128-8f26-43c1-934f-3008c23fbb95 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:37.577296 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06170128-8f26-43c1-934f-3008c23fbb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:37.595890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-1e239cff-2b53-4495-9a39-13b98f351363 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:37.611547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-1e239cff-2b53-4495-9a39-13b98f351363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:37.615169 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-06170128-8f26-43c1-934f-3008c23fbb95 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:37.616227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-06170128-8f26-43c1-934f-3008c23fbb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:37.728074 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1008/2013] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:37.752834 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e239cff-2b53-4495-9a39-13b98f351363 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:37.755240 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-1e239cff-2b53-4495-9a39-13b98f351363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:37.765488 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1006/2014] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:37.793144 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06170128-8f26-43c1-934f-3008c23fbb95 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:37.795194 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-06170128-8f26-43c1-934f-3008c23fbb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:37.853946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e239cff-2b53-4495-9a39-13b98f351363 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:37.858314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c48af0-a09a-42a6-9615-18ffceebfb89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:37.859232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c48af0-a09a-42a6-9615-18ffceebfb89 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:37.859559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c48af0-a09a-42a6-9615-18ffceebfb89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:37.860209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c48af0-a09a-42a6-9615-18ffceebfb89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:37.870097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c48af0-a09a-42a6-9615-18ffceebfb89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:37.871793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c48af0-a09a-42a6-9615-18ffceebfb89 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:37.887608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1009/2015] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:37.904274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06170128-8f26-43c1-934f-3008c23fbb95 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:37.908258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37934cff-559e-425e-bab0-6f6e7d8a9810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:37.909203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37934cff-559e-425e-bab0-6f6e7d8a9810 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:37.909505 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37934cff-559e-425e-bab0-6f6e7d8a9810 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:37.910616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37934cff-559e-425e-bab0-6f6e7d8a9810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:37.922665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37934cff-559e-425e-bab0-6f6e7d8a9810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:37.924220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37934cff-559e-425e-bab0-6f6e7d8a9810 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:37.945184 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1007/2016] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:38.621015 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c48af0-a09a-42a6-9615-18ffceebfb89 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:38.623184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-89c48af0-a09a-42a6-9615-18ffceebfb89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:38.637404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c48af0-a09a-42a6-9615-18ffceebfb89 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:38.645120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27c5d9-1def-4bb6-ae77-24c805254403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:38.656051 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27c5d9-1def-4bb6-ae77-24c805254403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:38.656343 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27c5d9-1def-4bb6-ae77-24c805254403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:38.671367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37934cff-559e-425e-bab0-6f6e7d8a9810 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:38.671856 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27c5d9-1def-4bb6-ae77-24c805254403 admin admin] PATH_INFO: `/v3/projects/fe1071f281434e18877a75fc8b6d6725` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:38.673417 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a27c5d9-1def-4bb6-ae77-24c805254403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe1071f281434e18877a75fc8b6d6725)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:38.674875 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-37934cff-559e-425e-bab0-6f6e7d8a9810 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:38.676938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a27c5d9-1def-4bb6-ae77-24c805254403 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:38.693146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37934cff-559e-425e-bab0-6f6e7d8a9810 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:38.698652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e9646-917c-420a-b47d-93c0ab9df5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:38.700206 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e9646-917c-420a-b47d-93c0ab9df5c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:38.700698 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e9646-917c-420a-b47d-93c0ab9df5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:38.704463 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e9646-917c-420a-b47d-93c0ab9df5c4 admin admin] PATH_INFO: `/v3/projects/091eed2ae7314b22be894ac0e3b3c199` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:38.707263 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e9646-917c-420a-b47d-93c0ab9df5c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=091eed2ae7314b22be894ac0e3b3c199)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:38.710449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e9646-917c-420a-b47d-93c0ab9df5c4 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:38.737926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-8a27c5d9-1def-4bb6-ae77-24c805254403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe1071f281434e18877a75fc8b6d6725', 'request_id': 'req-8a27c5d9-1def-4bb6-ae77-24c805254403'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:38.746719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-8a27c5d9-1def-4bb6-ae77-24c805254403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe1071f281434e18877a75fc8b6d6725', 'request_id': 'req-8a27c5d9-1def-4bb6-ae77-24c805254403'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:38.753021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-8a27c5d9-1def-4bb6-ae77-24c805254403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe1071f281434e18877a75fc8b6d6725', 'request_id': 'req-8a27c5d9-1def-4bb6-ae77-24c805254403'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:38.756005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:38.769785 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-8a27c5d9-1def-4bb6-ae77-24c805254403 admin admin] The token cache is being invalidate because project fe1071f281434e18877a75fc8b6d6725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:38.770085 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-8a27c5d9-1def-4bb6-ae77-24c805254403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:38.770876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-8a27c5d9-1def-4bb6-ae77-24c805254403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:38.772941 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1010/2017] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:38 2022] DELETE /identity/v3/projects/fe1071f281434e18877a75fc8b6d6725 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:38.776449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:38.776833 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27c5d9-1def-4bb6-ae77-24c805254403 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:38.778605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8a27c5d9-1def-4bb6-ae77-24c805254403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:38.782503 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-d89e9646-917c-420a-b47d-93c0ab9df5c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '091eed2ae7314b22be894ac0e3b3c199', 'request_id': 'req-d89e9646-917c-420a-b47d-93c0ab9df5c4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:38.793755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-d89e9646-917c-420a-b47d-93c0ab9df5c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '091eed2ae7314b22be894ac0e3b3c199', 'request_id': 'req-d89e9646-917c-420a-b47d-93c0ab9df5c4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:38.803909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-d89e9646-917c-420a-b47d-93c0ab9df5c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '091eed2ae7314b22be894ac0e3b3c199', 'request_id': 'req-d89e9646-917c-420a-b47d-93c0ab9df5c4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:38.818852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-d89e9646-917c-420a-b47d-93c0ab9df5c4 admin admin] The token cache is being invalidate because project 091eed2ae7314b22be894ac0e3b3c199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:38.819561 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-d89e9646-917c-420a-b47d-93c0ab9df5c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:38.819891 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-d89e9646-917c-420a-b47d-93c0ab9df5c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:38.821988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1008/2018] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:38 2022] DELETE /identity/v3/projects/091eed2ae7314b22be894ac0e3b3c199 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:38.826117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e9646-917c-420a-b47d-93c0ab9df5c4 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:38.827700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d89e9646-917c-420a-b47d-93c0ab9df5c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:38.857828 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27c5d9-1def-4bb6-ae77-24c805254403 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:38.930252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e9646-917c-420a-b47d-93c0ab9df5c4 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:38.936426 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4639362-cf92-4afb-9eb9-58331df25de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:38.937473 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4639362-cf92-4afb-9eb9-58331df25de9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:38.937826 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4639362-cf92-4afb-9eb9-58331df25de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:38.938167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4639362-cf92-4afb-9eb9-58331df25de9 admin admin] PATH_INFO: `/v3/users/1fd72a6c64cc4db3bcab684da0421581` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:38.940297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4639362-cf92-4afb-9eb9-58331df25de9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fd72a6c64cc4db3bcab684da0421581)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:38.942923 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4639362-cf92-4afb-9eb9-58331df25de9 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:38.970947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4317e-effc-48e4-ae97-afafc122b090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:38.973013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4317e-effc-48e4-ae97-afafc122b090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:38.973013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4317e-effc-48e4-ae97-afafc122b090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:38.973013 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4317e-effc-48e4-ae97-afafc122b090 admin admin] PATH_INFO: `/v3/users/ca58209b7ca4483895a0afa5560dc9d5` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:38.975151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4317e-effc-48e4-ae97-afafc122b090 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca58209b7ca4483895a0afa5560dc9d5)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:38.978032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4317e-effc-48e4-ae97-afafc122b090 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:39.030609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b4639362-cf92-4afb-9eb9-58331df25de9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fd72a6c64cc4db3bcab684da0421581'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.041114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e2c4317e-effc-48e4-ae97-afafc122b090 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca58209b7ca4483895a0afa5560dc9d5'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.046943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b4639362-cf92-4afb-9eb9-58331df25de9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fd72a6c64cc4db3bcab684da0421581'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.055373 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b4639362-cf92-4afb-9eb9-58331df25de9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fd72a6c64cc4db3bcab684da0421581'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.056416 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e2c4317e-effc-48e4-ae97-afafc122b090 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca58209b7ca4483895a0afa5560dc9d5'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.056878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b4639362-cf92-4afb-9eb9-58331df25de9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fd72a6c64cc4db3bcab684da0421581'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.057658 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b4639362-cf92-4afb-9eb9-58331df25de9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fd72a6c64cc4db3bcab684da0421581'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.067030 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e2c4317e-effc-48e4-ae97-afafc122b090 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca58209b7ca4483895a0afa5560dc9d5'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.067823 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e2c4317e-effc-48e4-ae97-afafc122b090 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca58209b7ca4483895a0afa5560dc9d5'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.068553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e2c4317e-effc-48e4-ae97-afafc122b090 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca58209b7ca4483895a0afa5560dc9d5'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.074818 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1009/2019] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:38 2022] DELETE /identity/v3/users/1fd72a6c64cc4db3bcab684da0421581 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:39.086918 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1011/2020] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:38 2022] DELETE /identity/v3/users/ca58209b7ca4483895a0afa5560dc9d5 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:39.421533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4639362-cf92-4afb-9eb9-58331df25de9 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:39.423658 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b4639362-cf92-4afb-9eb9-58331df25de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:39.459486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4317e-effc-48e4-ae97-afafc122b090 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:39.461632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-e2c4317e-effc-48e4-ae97-afafc122b090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:39.516938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:39.552016 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:39.569005 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4639362-cf92-4afb-9eb9-58331df25de9 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:39.663226 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f46645-222e-4dfe-9ed6-f2c72b35dc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:39.666323 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4317e-effc-48e4-ae97-afafc122b090 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:39.667368 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f46645-222e-4dfe-9ed6-f2c72b35dc76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:39.667716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f46645-222e-4dfe-9ed6-f2c72b35dc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:39.667981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f46645-222e-4dfe-9ed6-f2c72b35dc76 admin admin] PATH_INFO: `/v3/projects/b65615acf98642679d6b605e205aab04` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:39.669459 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f46645-222e-4dfe-9ed6-f2c72b35dc76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b65615acf98642679d6b605e205aab04)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:39.671674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f46645-222e-4dfe-9ed6-f2c72b35dc76 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:39.686824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6226bd4-65c4-4538-a301-8d9c88fe1682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:39.687807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6226bd4-65c4-4538-a301-8d9c88fe1682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:39.688093 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6226bd4-65c4-4538-a301-8d9c88fe1682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:39.688373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6226bd4-65c4-4538-a301-8d9c88fe1682 admin admin] PATH_INFO: `/v3/projects/e24ef305d5c9427fbba9c9da6c56d4c0` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:39.692778 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6226bd4-65c4-4538-a301-8d9c88fe1682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e24ef305d5c9427fbba9c9da6c56d4c0)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:39.710203 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6226bd4-65c4-4538-a301-8d9c88fe1682 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:39.762564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-56f46645-222e-4dfe-9ed6-f2c72b35dc76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b65615acf98642679d6b605e205aab04', 'request_id': 'req-56f46645-222e-4dfe-9ed6-f2c72b35dc76'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.781773 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-56f46645-222e-4dfe-9ed6-f2c72b35dc76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b65615acf98642679d6b605e205aab04', 'request_id': 'req-56f46645-222e-4dfe-9ed6-f2c72b35dc76'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.794140 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-56f46645-222e-4dfe-9ed6-f2c72b35dc76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b65615acf98642679d6b605e205aab04', 'request_id': 'req-56f46645-222e-4dfe-9ed6-f2c72b35dc76'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.804434 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-a6226bd4-65c4-4538-a301-8d9c88fe1682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e24ef305d5c9427fbba9c9da6c56d4c0', 'request_id': 'req-a6226bd4-65c4-4538-a301-8d9c88fe1682'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.811568 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-56f46645-222e-4dfe-9ed6-f2c72b35dc76 admin admin] The token cache is being invalidate because project b65615acf98642679d6b605e205aab04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:39.812119 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-56f46645-222e-4dfe-9ed6-f2c72b35dc76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.813156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-56f46645-222e-4dfe-9ed6-f2c72b35dc76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.813438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-a6226bd4-65c4-4538-a301-8d9c88fe1682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e24ef305d5c9427fbba9c9da6c56d4c0', 'request_id': 'req-a6226bd4-65c4-4538-a301-8d9c88fe1682'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.818275 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1010/2021] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:39 2022] DELETE /identity/v3/projects/b65615acf98642679d6b605e205aab04 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:39.819956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-a6226bd4-65c4-4538-a301-8d9c88fe1682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e24ef305d5c9427fbba9c9da6c56d4c0', 'request_id': 'req-a6226bd4-65c4-4538-a301-8d9c88fe1682'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.820242 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ae141-4336-4f52-bfbf-960cf4afa298 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:39.820746 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ae141-4336-4f52-bfbf-960cf4afa298 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:39.821292 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ae141-4336-4f52-bfbf-960cf4afa298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:39.843199 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-a6226bd4-65c4-4538-a301-8d9c88fe1682 admin admin] The token cache is being invalidate because project e24ef305d5c9427fbba9c9da6c56d4c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:39.843543 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-a6226bd4-65c4-4538-a301-8d9c88fe1682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.844175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-a6226bd4-65c4-4538-a301-8d9c88fe1682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:39.847058 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1012/2022] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:39 2022] DELETE /identity/v3/projects/e24ef305d5c9427fbba9c9da6c56d4c0 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:39.847217 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-167ae141-4336-4f52-bfbf-960cf4afa298 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:39.848631 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-167ae141-4336-4f52-bfbf-960cf4afa298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:39.849984 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6226bd4-65c4-4538-a301-8d9c88fe1682 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:39.852689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a6226bd4-65c4-4538-a301-8d9c88fe1682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:39.948037 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1011/2023] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:39.955801 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255799e-b18b-40c7-bad1-628b634f7993 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:39.956089 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255799e-b18b-40c7-bad1-628b634f7993 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:39.956363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255799e-b18b-40c7-bad1-628b634f7993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:39.966375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6226bd4-65c4-4538-a301-8d9c88fe1682 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:39.971174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf172a2f-e138-474a-854a-ac6af9228631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:39.972067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf172a2f-e138-474a-854a-ac6af9228631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:39.972351 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf172a2f-e138-474a-854a-ac6af9228631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:39.972630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf172a2f-e138-474a-854a-ac6af9228631 admin admin] PATH_INFO: `/v3/users/93905ff928f04bafa54542635ddab9eb` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:39.973926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf172a2f-e138-474a-854a-ac6af9228631 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93905ff928f04bafa54542635ddab9eb)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:39.974690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-e255799e-b18b-40c7-bad1-628b634f7993 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:39.975926 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e255799e-b18b-40c7-bad1-628b634f7993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:39.976042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf172a2f-e138-474a-854a-ac6af9228631 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:39.983914 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1012/2024] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:44:39.988482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255799e-b18b-40c7-bad1-628b634f7993 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:39.998393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255799e-b18b-40c7-bad1-628b634f7993 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:40.003914 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b620209-bdc7-4f4e-8188-0a83b286b776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:40.004680 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b620209-bdc7-4f4e-8188-0a83b286b776 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:40.005021 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b620209-bdc7-4f4e-8188-0a83b286b776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:40.005145 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b620209-bdc7-4f4e-8188-0a83b286b776 admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:40.006332 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b620209-bdc7-4f4e-8188-0a83b286b776 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:44:40.006332 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b620209-bdc7-4f4e-8188-0a83b286b776 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:40.007883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b620209-bdc7-4f4e-8188-0a83b286b776 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:40.024537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1013/2025] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:44:40.025861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b620209-bdc7-4f4e-8188-0a83b286b776 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:40.030457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-bf172a2f-e138-474a-854a-ac6af9228631 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93905ff928f04bafa54542635ddab9eb'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:40.036404 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b620209-bdc7-4f4e-8188-0a83b286b776 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:40.037383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:40.043215 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107ea6d-b0e8-40f6-b288-a93eab939b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:40.044599 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107ea6d-b0e8-40f6-b288-a93eab939b83 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:40.047303 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107ea6d-b0e8-40f6-b288-a93eab939b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:40.047592 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107ea6d-b0e8-40f6-b288-a93eab939b83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:40.048403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107ea6d-b0e8-40f6-b288-a93eab939b83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:40.051651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107ea6d-b0e8-40f6-b288-a93eab939b83 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:40.051840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-bf172a2f-e138-474a-854a-ac6af9228631 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93905ff928f04bafa54542635ddab9eb'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:40.066444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:40.073481 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-bf172a2f-e138-474a-854a-ac6af9228631 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93905ff928f04bafa54542635ddab9eb'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:40.074542 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-bf172a2f-e138-474a-854a-ac6af9228631 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93905ff928f04bafa54542635ddab9eb'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:40.075583 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-bf172a2f-e138-474a-854a-ac6af9228631 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93905ff928f04bafa54542635ddab9eb'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:40.097398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1013/2026] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:39 2022] DELETE /identity/v3/users/93905ff928f04bafa54542635ddab9eb => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:40.099921 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1014/2027] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:40 2022] POST /identity/v3/projects => generated 367 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:44:40.108425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf172a2f-e138-474a-854a-ac6af9228631 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:40.110103 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-bf172a2f-e138-474a-854a-ac6af9228631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:40.204535 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf172a2f-e138-474a-854a-ac6af9228631 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:40.240480 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107ea6d-b0e8-40f6-b288-a93eab939b83 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:40.242817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f107ea6d-b0e8-40f6-b288-a93eab939b83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:40.301570 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99188875-44de-4f29-a4b4-caab6a2e8029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:40.303905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99188875-44de-4f29-a4b4-caab6a2e8029 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:40.303905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99188875-44de-4f29-a4b4-caab6a2e8029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:40.303905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99188875-44de-4f29-a4b4-caab6a2e8029 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:40.304514 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99188875-44de-4f29-a4b4-caab6a2e8029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:40.306702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99188875-44de-4f29-a4b4-caab6a2e8029 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:40.338937 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1014/2028] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:40 2022] POST /identity/v3/users => generated 284 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:40.343151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99188875-44de-4f29-a4b4-caab6a2e8029 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:40.351554 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107ea6d-b0e8-40f6-b288-a93eab939b83 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:40.353976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99188875-44de-4f29-a4b4-caab6a2e8029 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:40.356971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824c4f4-47af-44bb-b9ca-b2e3b88b4f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:40.357939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824c4f4-47af-44bb-b9ca-b2e3b88b4f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:40.358226 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824c4f4-47af-44bb-b9ca-b2e3b88b4f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:40.358506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824c4f4-47af-44bb-b9ca-b2e3b88b4f0e admin admin] PATH_INFO: `/v3/users/f21724b0629a4ba59583ac9cee837506` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:40.359081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e06a5f-7326-4da0-b5db-99d34ae5ce84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:40.359860 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e06a5f-7326-4da0-b5db-99d34ae5ce84 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:40.360250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e06a5f-7326-4da0-b5db-99d34ae5ce84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:40.360534 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e06a5f-7326-4da0-b5db-99d34ae5ce84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:40.360880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824c4f4-47af-44bb-b9ca-b2e3b88b4f0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f21724b0629a4ba59583ac9cee837506)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:40.361644 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e06a5f-7326-4da0-b5db-99d34ae5ce84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:40.363170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e06a5f-7326-4da0-b5db-99d34ae5ce84 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:40.363448 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824c4f4-47af-44bb-b9ca-b2e3b88b4f0e admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:40.374217 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1015/2029] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:40.379098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e06a5f-7326-4da0-b5db-99d34ae5ce84 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:40.392376 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e06a5f-7326-4da0-b5db-99d34ae5ce84 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:40.397982 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde997e-33db-47ac-bf97-2c74897ef96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:40.398721 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde997e-33db-47ac-bf97-2c74897ef96e admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:40.399040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde997e-33db-47ac-bf97-2c74897ef96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:40.399376 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde997e-33db-47ac-bf97-2c74897ef96e admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:40.400074 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde997e-33db-47ac-bf97-2c74897ef96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:40.402359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde997e-33db-47ac-bf97-2c74897ef96e admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:40.414505 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1016/2030] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:44:40.418874 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde997e-33db-47ac-bf97-2c74897ef96e admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:40.432161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde997e-33db-47ac-bf97-2c74897ef96e admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:40.437378 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3824c4f4-47af-44bb-b9ca-b2e3b88b4f0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f21724b0629a4ba59583ac9cee837506'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:40.440052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74148d09-59db-428c-957b-771bda9ab598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:40.441001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74148d09-59db-428c-957b-771bda9ab598 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:40.441258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74148d09-59db-428c-957b-771bda9ab598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:40.441501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74148d09-59db-428c-957b-771bda9ab598 admin admin] PATH_INFO: `/v3/projects/fdd73263047847afa95bac79ae5fdf0a/users/d379b2d054df462bb741f8c7d72df7ac/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:40.454979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3824c4f4-47af-44bb-b9ca-b2e3b88b4f0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f21724b0629a4ba59583ac9cee837506'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:40.459266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74148d09-59db-428c-957b-771bda9ab598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd73263047847afa95bac79ae5fdf0a, user_id=d379b2d054df462bb741f8c7d72df7ac, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:40.462057 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74148d09-59db-428c-957b-771bda9ab598 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:40.463399 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3824c4f4-47af-44bb-b9ca-b2e3b88b4f0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f21724b0629a4ba59583ac9cee837506'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:40.464225 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3824c4f4-47af-44bb-b9ca-b2e3b88b4f0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f21724b0629a4ba59583ac9cee837506'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:40.464938 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3824c4f4-47af-44bb-b9ca-b2e3b88b4f0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f21724b0629a4ba59583ac9cee837506'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:40.485243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1017/2031] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:40 2022] PUT /identity/v3/projects/fdd73263047847afa95bac79ae5fdf0a/users/d379b2d054df462bb741f8c7d72df7ac/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:40.486381 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74148d09-59db-428c-957b-771bda9ab598 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:40.489688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1015/2032] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:40 2022] DELETE /identity/v3/users/f21724b0629a4ba59583ac9cee837506 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:40.491290 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-74148d09-59db-428c-957b-771bda9ab598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:40.500694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12600642-889a-499a-94fd-6b25307a0b6a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:40.500989 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12600642-889a-499a-94fd-6b25307a0b6a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:40.501306 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12600642-889a-499a-94fd-6b25307a0b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:40.519046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:40.530665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-12600642-889a-499a-94fd-6b25307a0b6a None None] MFA Rules not processed for user `d379b2d054df462bb741f8c7d72df7ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:40.532381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-12600642-889a-499a-94fd-6b25307a0b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:40.567839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74148d09-59db-428c-957b-771bda9ab598 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:40.595630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:40.601784 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1016/2033] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:40 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:40.607977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d94bf-8f4d-4042-81f9-58ce1b8f7612 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:40.607977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d94bf-8f4d-4042-81f9-58ce1b8f7612 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:40.610706 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d94bf-8f4d-4042-81f9-58ce1b8f7612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:40.628963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-612d94bf-8f4d-4042-81f9-58ce1b8f7612 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:40.629511 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993ab03-28cc-4b8e-8a59-e37104828e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:40.630269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-612d94bf-8f4d-4042-81f9-58ce1b8f7612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:40.630506 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993ab03-28cc-4b8e-8a59-e37104828e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:40.630885 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993ab03-28cc-4b8e-8a59-e37104828e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:40.631302 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993ab03-28cc-4b8e-8a59-e37104828e43 admin admin] PATH_INFO: `/v3/projects/55fa311fd61446499bf25fdc96cf9f7c` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:40.632605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993ab03-28cc-4b8e-8a59-e37104828e43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55fa311fd61446499bf25fdc96cf9f7c)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:40.634686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993ab03-28cc-4b8e-8a59-e37104828e43 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:40.638885 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1017/2034] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:40.639582 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d94bf-8f4d-4042-81f9-58ce1b8f7612 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:40.640886 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-612d94bf-8f4d-4042-81f9-58ce1b8f7612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:40.704273 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1993ab03-28cc-4b8e-8a59-e37104828e43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55fa311fd61446499bf25fdc96cf9f7c', 'request_id': 'req-1993ab03-28cc-4b8e-8a59-e37104828e43'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:40.708005 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d94bf-8f4d-4042-81f9-58ce1b8f7612 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:40.712233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1993ab03-28cc-4b8e-8a59-e37104828e43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55fa311fd61446499bf25fdc96cf9f7c', 'request_id': 'req-1993ab03-28cc-4b8e-8a59-e37104828e43'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:40.719676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1993ab03-28cc-4b8e-8a59-e37104828e43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55fa311fd61446499bf25fdc96cf9f7c', 'request_id': 'req-1993ab03-28cc-4b8e-8a59-e37104828e43'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:40.738990 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1993ab03-28cc-4b8e-8a59-e37104828e43 admin admin] The token cache is being invalidate because project 55fa311fd61446499bf25fdc96cf9f7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:40.739411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1993ab03-28cc-4b8e-8a59-e37104828e43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:40.740476 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1993ab03-28cc-4b8e-8a59-e37104828e43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:40.747212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1018/2035] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:40 2022] DELETE /identity/v3/projects/55fa311fd61446499bf25fdc96cf9f7c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:40.747212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993ab03-28cc-4b8e-8a59-e37104828e43 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:40.747212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-1993ab03-28cc-4b8e-8a59-e37104828e43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:40.767888 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f2f17-f86b-44f0-be71-f2a9e0bdf6d6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2707c6967a0a419a98ae142ff1d1a961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:40.768666 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f2f17-f86b-44f0-be71-f2a9e0bdf6d6 service glance] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:40.769025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f2f17-f86b-44f0-be71-f2a9e0bdf6d6 service glance] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:40.769381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f2f17-f86b-44f0-be71-f2a9e0bdf6d6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:40.771689 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-124f2f17-f86b-44f0-be71-f2a9e0bdf6d6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:40.823394 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993ab03-28cc-4b8e-8a59-e37104828e43 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:40.828130 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ed183-a64d-48d8-a5ea-00b5a36ce3bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:40.829219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ed183-a64d-48d8-a5ea-00b5a36ce3bc service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:40.829219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ed183-a64d-48d8-a5ea-00b5a36ce3bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:40.829450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ed183-a64d-48d8-a5ea-00b5a36ce3bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:40.831553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-fd3ed183-a64d-48d8-a5ea-00b5a36ce3bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:40.846573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:40.855849 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f2f17-f86b-44f0-be71-f2a9e0bdf6d6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:40.857564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f2f17-f86b-44f0-be71-f2a9e0bdf6d6 service glance] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:40.867657 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1018/2036] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:40 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:40.871078 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aac9d-d9cc-4003-9149-924a829e7f89 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:40.871602 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aac9d-d9cc-4003-9149-924a829e7f89 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:40.871973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38aac9d-d9cc-4003-9149-924a829e7f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:40.898891 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-f38aac9d-d9cc-4003-9149-924a829e7f89 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:40.900362 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f38aac9d-d9cc-4003-9149-924a829e7f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:40.957250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ed183-a64d-48d8-a5ea-00b5a36ce3bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:40.966116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ed183-a64d-48d8-a5ea-00b5a36ce3bc service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:41.028500 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1019/2037] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:41.110309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1019/2038] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:41.118531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b763a1-d55e-46a0-b9b5-89b8480301f4 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:41.118896 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b763a1-d55e-46a0-b9b5-89b8480301f4 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:41.120421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b763a1-d55e-46a0-b9b5-89b8480301f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:41.145142 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-30b763a1-d55e-46a0-b9b5-89b8480301f4 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:41.149063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-30b763a1-d55e-46a0-b9b5-89b8480301f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:41.168640 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1020/2039] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:41.173031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38aac9d-d9cc-4003-9149-924a829e7f89 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:41.181863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:41.196995 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38aac9d-d9cc-4003-9149-924a829e7f89 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:41.210165 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143df96-a3f0-4b12-b50c-b5adcf570a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:41.211364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143df96-a3f0-4b12-b50c-b5adcf570a2a admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:41.211364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143df96-a3f0-4b12-b50c-b5adcf570a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:41.211547 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143df96-a3f0-4b12-b50c-b5adcf570a2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:41.212333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143df96-a3f0-4b12-b50c-b5adcf570a2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:44:41.212652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143df96-a3f0-4b12-b50c-b5adcf570a2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:41.214394 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143df96-a3f0-4b12-b50c-b5adcf570a2a admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:41.229234 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1020/2040] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:44:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:44:41.233501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b763a1-d55e-46a0-b9b5-89b8480301f4 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:41.249518 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b763a1-d55e-46a0-b9b5-89b8480301f4 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:41.256358 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c4b1b-ecce-4bff-944c-5c231b44c97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:41.257425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c4b1b-ecce-4bff-944c-5c231b44c97a admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:41.257876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c4b1b-ecce-4bff-944c-5c231b44c97a admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:41.258272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c4b1b-ecce-4bff-944c-5c231b44c97a admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:41.259425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c4b1b-ecce-4bff-944c-5c231b44c97a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:41.262659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c4b1b-ecce-4bff-944c-5c231b44c97a admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:41.312710 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1021/2041] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:41 2022] POST /identity/v3/projects => generated 375 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:41.317250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143df96-a3f0-4b12-b50c-b5adcf570a2a admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:41.343621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143df96-a3f0-4b12-b50c-b5adcf570a2a admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:41.449180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36003cc2-fa22-439b-821c-38253804b714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:41.449180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36003cc2-fa22-439b-821c-38253804b714 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:41.449180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36003cc2-fa22-439b-821c-38253804b714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:41.449180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36003cc2-fa22-439b-821c-38253804b714 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:41.450833 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36003cc2-fa22-439b-821c-38253804b714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:41.453124 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36003cc2-fa22-439b-821c-38253804b714 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:41.489034 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1021/2042] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:41 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:44:41.494534 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c4b1b-ecce-4bff-944c-5c231b44c97a admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:41.508841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c4b1b-ecce-4bff-944c-5c231b44c97a admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:41.521450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94848fc-7337-44b1-af56-8e109c2337bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:41.539950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94848fc-7337-44b1-af56-8e109c2337bd admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:41.540261 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94848fc-7337-44b1-af56-8e109c2337bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:41.540552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94848fc-7337-44b1-af56-8e109c2337bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:41.541885 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94848fc-7337-44b1-af56-8e109c2337bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:41.545307 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94848fc-7337-44b1-af56-8e109c2337bd admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:41.580690 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1022/2043] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:41.585290 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36003cc2-fa22-439b-821c-38253804b714 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:41.606466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94848fc-7337-44b1-af56-8e109c2337bd admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:41.606836 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36003cc2-fa22-439b-821c-38253804b714 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:41.610359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d94848fc-7337-44b1-af56-8e109c2337bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:41.622316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6974f-f63a-4318-a8c7-512a795ae896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:41.623907 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6974f-f63a-4318-a8c7-512a795ae896 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:41.624260 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6974f-f63a-4318-a8c7-512a795ae896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:41.624558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6974f-f63a-4318-a8c7-512a795ae896 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:41.626291 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6974f-f63a-4318-a8c7-512a795ae896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:41.636409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6974f-f63a-4318-a8c7-512a795ae896 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:41.656073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94848fc-7337-44b1-af56-8e109c2337bd admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:41.673355 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1022/2044] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:41.674715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5e986-0c51-4604-abd3-7ade7e0dad08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:41.675630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5e986-0c51-4604-abd3-7ade7e0dad08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:41.675911 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5e986-0c51-4604-abd3-7ade7e0dad08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:41.676517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5e986-0c51-4604-abd3-7ade7e0dad08 admin admin] PATH_INFO: `/v3/projects/4ff44cd9d5c04fb5bf58db66d2a7e16b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:41.678047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c5e986-0c51-4604-abd3-7ade7e0dad08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ff44cd9d5c04fb5bf58db66d2a7e16b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:41.680028 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c5e986-0c51-4604-abd3-7ade7e0dad08 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:41.682425 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6974f-f63a-4318-a8c7-512a795ae896 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:41.703798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6974f-f63a-4318-a8c7-512a795ae896 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:41.716408 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c0d21-bf67-4a48-8d77-6f2f94c923bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:41.717625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c0d21-bf67-4a48-8d77-6f2f94c923bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:41.718098 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c0d21-bf67-4a48-8d77-6f2f94c923bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:41.718379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c0d21-bf67-4a48-8d77-6f2f94c923bc admin admin] PATH_INFO: `/v3/projects/8da79944da71422c8d3eb6712d25f23b/users/2ac9950281cd45439311905a03527f57/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:41.746643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c0d21-bf67-4a48-8d77-6f2f94c923bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8da79944da71422c8d3eb6712d25f23b, user_id=2ac9950281cd45439311905a03527f57, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:41.752792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c0d21-bf67-4a48-8d77-6f2f94c923bc admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:41.785920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1023/2045] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:41 2022] PUT /identity/v3/projects/8da79944da71422c8d3eb6712d25f23b/users/2ac9950281cd45439311905a03527f57/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:41.799517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-33c5e986-0c51-4604-abd3-7ade7e0dad08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ff44cd9d5c04fb5bf58db66d2a7e16b', 'request_id': 'req-33c5e986-0c51-4604-abd3-7ade7e0dad08'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:41.804225 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86640ecb-b23e-4092-9dbc-6a0be4851c8a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:41.804631 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86640ecb-b23e-4092-9dbc-6a0be4851c8a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:41.805464 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86640ecb-b23e-4092-9dbc-6a0be4851c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:41.824270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-33c5e986-0c51-4604-abd3-7ade7e0dad08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ff44cd9d5c04fb5bf58db66d2a7e16b', 'request_id': 'req-33c5e986-0c51-4604-abd3-7ade7e0dad08'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:41.836844 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-33c5e986-0c51-4604-abd3-7ade7e0dad08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ff44cd9d5c04fb5bf58db66d2a7e16b', 'request_id': 'req-33c5e986-0c51-4604-abd3-7ade7e0dad08'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:41.851528 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-86640ecb-b23e-4092-9dbc-6a0be4851c8a None None] MFA Rules not processed for user `2ac9950281cd45439311905a03527f57`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:41.853199 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-86640ecb-b23e-4092-9dbc-6a0be4851c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:41.861323 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-33c5e986-0c51-4604-abd3-7ade7e0dad08 admin admin] The token cache is being invalidate because project 4ff44cd9d5c04fb5bf58db66d2a7e16b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:41.861619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-33c5e986-0c51-4604-abd3-7ade7e0dad08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:41.862559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-33c5e986-0c51-4604-abd3-7ade7e0dad08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:41.867418 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1023/2046] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:41 2022] DELETE /identity/v3/projects/4ff44cd9d5c04fb5bf58db66d2a7e16b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:41.868617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5e986-0c51-4604-abd3-7ade7e0dad08 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:41.869989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-33c5e986-0c51-4604-abd3-7ade7e0dad08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:41.928589 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:41.937534 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1024/2047] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:44:41.942857 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86640ecb-b23e-4092-9dbc-6a0be4851c8a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:41.944740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-86640ecb-b23e-4092-9dbc-6a0be4851c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:41.946356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5e986-0c51-4604-abd3-7ade7e0dad08 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:41.985730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:42.019075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86640ecb-b23e-4092-9dbc-6a0be4851c8a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:42.020151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a9ed1-d7e6-4d0e-9e84-c88414615bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:42.021198 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a9ed1-d7e6-4d0e-9e84-c88414615bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:42.022078 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a9ed1-d7e6-4d0e-9e84-c88414615bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:42.022239 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a9ed1-d7e6-4d0e-9e84-c88414615bfd admin admin] PATH_INFO: `/v3/users/4462b522f2614c83a8e42a7b2dc7a2e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:42.024139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a9ed1-d7e6-4d0e-9e84-c88414615bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4462b522f2614c83a8e42a7b2dc7a2e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:42.027120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28a9ed1-d7e6-4d0e-9e84-c88414615bfd admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:42.034073 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d748d5-cd81-4ea1-b7a1-7389b291fe7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:42.035158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d748d5-cd81-4ea1-b7a1-7389b291fe7a admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:42.039654 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d748d5-cd81-4ea1-b7a1-7389b291fe7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:42.039919 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d748d5-cd81-4ea1-b7a1-7389b291fe7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:42.041315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d748d5-cd81-4ea1-b7a1-7389b291fe7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:42.044683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d748d5-cd81-4ea1-b7a1-7389b291fe7a admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:42.101105 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1025/2048] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:41 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:44:42.103250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d748d5-cd81-4ea1-b7a1-7389b291fe7a admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:42.131067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f28a9ed1-d7e6-4d0e-9e84-c88414615bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4462b522f2614c83a8e42a7b2dc7a2e8'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:42.158065 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f28a9ed1-d7e6-4d0e-9e84-c88414615bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4462b522f2614c83a8e42a7b2dc7a2e8'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:42.167475 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f28a9ed1-d7e6-4d0e-9e84-c88414615bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4462b522f2614c83a8e42a7b2dc7a2e8'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:42.168576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f28a9ed1-d7e6-4d0e-9e84-c88414615bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4462b522f2614c83a8e42a7b2dc7a2e8'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:42.169446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-f28a9ed1-d7e6-4d0e-9e84-c88414615bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4462b522f2614c83a8e42a7b2dc7a2e8'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:42.196906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1024/2049] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:41 2022] DELETE /identity/v3/users/4462b522f2614c83a8e42a7b2dc7a2e8 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:42.211035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d748d5-cd81-4ea1-b7a1-7389b291fe7a admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:42.280996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f0442-e4bf-480f-835a-e143565f8319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:42.281814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f0442-e4bf-480f-835a-e143565f8319 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:42.282179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f0442-e4bf-480f-835a-e143565f8319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:42.282543 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f0442-e4bf-480f-835a-e143565f8319 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:42.283648 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f0442-e4bf-480f-835a-e143565f8319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:42.286241 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f0442-e4bf-480f-835a-e143565f8319 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:42.311134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1026/2050] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:42 2022] POST /identity/v3/users => generated 295 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:42.312447 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a9ed1-d7e6-4d0e-9e84-c88414615bfd admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:42.313885 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f28a9ed1-d7e6-4d0e-9e84-c88414615bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:42.326454 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a9ed1-d7e6-4d0e-9e84-c88414615bfd admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:42.340822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f38ef3-9294-4a20-b731-0797c83dbd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:42.341577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f38ef3-9294-4a20-b731-0797c83dbd2d admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:42.341861 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f38ef3-9294-4a20-b731-0797c83dbd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:42.342213 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f38ef3-9294-4a20-b731-0797c83dbd2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:42.342981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f38ef3-9294-4a20-b731-0797c83dbd2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:42.344910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f38ef3-9294-4a20-b731-0797c83dbd2d admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:42.359390 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1025/2051] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:42.363286 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f0442-e4bf-480f-835a-e143565f8319 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:42.384824 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f0442-e4bf-480f-835a-e143565f8319 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:42.390276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba6382-4a8c-486b-bcb6-32b422e35d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:42.391373 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba6382-4a8c-486b-bcb6-32b422e35d67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:42.391751 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba6382-4a8c-486b-bcb6-32b422e35d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:42.392096 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba6382-4a8c-486b-bcb6-32b422e35d67 admin admin] PATH_INFO: `/v3/projects/bcb1e6e62e36473bb857396bab3d35e2/users/79c6a4ede8c8421fa10c68a8bffa858e/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:42.407800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba6382-4a8c-486b-bcb6-32b422e35d67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb1e6e62e36473bb857396bab3d35e2, user_id=79c6a4ede8c8421fa10c68a8bffa858e, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:42.410870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba6382-4a8c-486b-bcb6-32b422e35d67 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:42.425367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1027/2052] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:42 2022] PUT /identity/v3/projects/bcb1e6e62e36473bb857396bab3d35e2/users/79c6a4ede8c8421fa10c68a8bffa858e/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:42.429773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f38ef3-9294-4a20-b731-0797c83dbd2d admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:42.440062 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f38ef3-9294-4a20-b731-0797c83dbd2d admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:42.508743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1ecec-9079-46a4-ade4-19fbb50c8eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:42.509586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1ecec-9079-46a4-ade4-19fbb50c8eff admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:42.509853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1ecec-9079-46a4-ade4-19fbb50c8eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:42.510151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1ecec-9079-46a4-ade4-19fbb50c8eff admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:42.510926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1ecec-9079-46a4-ade4-19fbb50c8eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:42.513443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1ecec-9079-46a4-ade4-19fbb50c8eff admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:42.530092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1026/2053] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:42.532603 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba6382-4a8c-486b-bcb6-32b422e35d67 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:42.544657 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba6382-4a8c-486b-bcb6-32b422e35d67 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:42.552955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92387a58-777d-45b0-a1a4-731f20f0b16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:42.554680 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92387a58-777d-45b0-a1a4-731f20f0b16b admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:42.555117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92387a58-777d-45b0-a1a4-731f20f0b16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:42.556126 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92387a58-777d-45b0-a1a4-731f20f0b16b admin admin] PATH_INFO: `/v3/projects/bcb1e6e62e36473bb857396bab3d35e2/users/79c6a4ede8c8421fa10c68a8bffa858e/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:42.560919 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92387a58-777d-45b0-a1a4-731f20f0b16b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb1e6e62e36473bb857396bab3d35e2, user_id=79c6a4ede8c8421fa10c68a8bffa858e, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:42.564662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92387a58-777d-45b0-a1a4-731f20f0b16b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:42.590401 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1ecec-9079-46a4-ade4-19fbb50c8eff admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:42.592466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d0a1ecec-9079-46a4-ade4-19fbb50c8eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:42.599272 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1028/2054] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:42 2022] PUT /identity/v3/projects/bcb1e6e62e36473bb857396bab3d35e2/users/79c6a4ede8c8421fa10c68a8bffa858e/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:42.602859 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:42.610091 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1568ea-3416-4cdd-9e5a-e5f15a6d5575 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:42.610374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1568ea-3416-4cdd-9e5a-e5f15a6d5575 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:42.610655 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1568ea-3416-4cdd-9e5a-e5f15a6d5575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:42.613125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1ecec-9079-46a4-ade4-19fbb50c8eff admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:42.648315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-cb1568ea-3416-4cdd-9e5a-e5f15a6d5575 None None] MFA Rules not processed for user `79c6a4ede8c8421fa10c68a8bffa858e`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:42.648315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-cb1568ea-3416-4cdd-9e5a-e5f15a6d5575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:42.653843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:42.682190 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5d615-6722-4af6-a5bf-12c3ed569282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:42.683116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5d615-6722-4af6-a5bf-12c3ed569282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:42.683401 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5d615-6722-4af6-a5bf-12c3ed569282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:42.683691 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5d615-6722-4af6-a5bf-12c3ed569282 admin admin] PATH_INFO: `/v3/projects/486153bad1504f1fa4190758e8a66668` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:42.684899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b5d615-6722-4af6-a5bf-12c3ed569282 admin admin] RBAC: Authorizing `identity:delete_project(project_id=486153bad1504f1fa4190758e8a66668)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:42.686851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b5d615-6722-4af6-a5bf-12c3ed569282 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:42.737448 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1029/2055] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:44:42.742485 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-60b5d615-6722-4af6-a5bf-12c3ed569282 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '486153bad1504f1fa4190758e8a66668', 'request_id': 'req-60b5d615-6722-4af6-a5bf-12c3ed569282'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:42.749096 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-60b5d615-6722-4af6-a5bf-12c3ed569282 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '486153bad1504f1fa4190758e8a66668', 'request_id': 'req-60b5d615-6722-4af6-a5bf-12c3ed569282'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:42.751712 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1568ea-3416-4cdd-9e5a-e5f15a6d5575 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:42.753198 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-cb1568ea-3416-4cdd-9e5a-e5f15a6d5575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:42.755760 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-60b5d615-6722-4af6-a5bf-12c3ed569282 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '486153bad1504f1fa4190758e8a66668', 'request_id': 'req-60b5d615-6722-4af6-a5bf-12c3ed569282'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:42.776031 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-60b5d615-6722-4af6-a5bf-12c3ed569282 admin admin] The token cache is being invalidate because project 486153bad1504f1fa4190758e8a66668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:42.776370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-60b5d615-6722-4af6-a5bf-12c3ed569282 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:42.777430 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-60b5d615-6722-4af6-a5bf-12c3ed569282 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:42.778566 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1027/2056] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:42 2022] DELETE /identity/v3/projects/486153bad1504f1fa4190758e8a66668 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:42.786135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6406a-f493-4f3d-8cf8-9911f6304b4c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:42.786432 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6406a-f493-4f3d-8cf8-9911f6304b4c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:42.786708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6406a-f493-4f3d-8cf8-9911f6304b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:42.806587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-8dc6406a-f493-4f3d-8cf8-9911f6304b4c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:42.807817 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8dc6406a-f493-4f3d-8cf8-9911f6304b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:42.820893 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1568ea-3416-4cdd-9e5a-e5f15a6d5575 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:42.899587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858e962-0165-4d23-b8d2-d8d180ce71d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:42.900466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858e962-0165-4d23-b8d2-d8d180ce71d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:42.900941 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858e962-0165-4d23-b8d2-d8d180ce71d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:42.901230 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858e962-0165-4d23-b8d2-d8d180ce71d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:42.903320 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d858e962-0165-4d23-b8d2-d8d180ce71d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:42.913190 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1028/2057] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:44:42.922019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafef73-92ba-4b58-b805-b7e9830f9947 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:42.922316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafef73-92ba-4b58-b805-b7e9830f9947 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:42.922601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fafef73-92ba-4b58-b805-b7e9830f9947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:42.955875 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-7fafef73-92ba-4b58-b805-b7e9830f9947 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:42.957104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7fafef73-92ba-4b58-b805-b7e9830f9947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:42.967645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1029/2058] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:44:42.972514 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafef73-92ba-4b58-b805-b7e9830f9947 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:42.981652 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fafef73-92ba-4b58-b805-b7e9830f9947 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:42.986491 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2fc22-42ed-4d8a-8527-f621fe4041f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:42.987195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2fc22-42ed-4d8a-8527-f621fe4041f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:42.987476 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2fc22-42ed-4d8a-8527-f621fe4041f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:42.987749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2fc22-42ed-4d8a-8527-f621fe4041f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:42.988619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e2fc22-42ed-4d8a-8527-f621fe4041f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:44:42.988924 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e2fc22-42ed-4d8a-8527-f621fe4041f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:42.990574 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e2fc22-42ed-4d8a-8527-f621fe4041f8 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:43.000744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858e962-0165-4d23-b8d2-d8d180ce71d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:43.002531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858e962-0165-4d23-b8d2-d8d180ce71d4 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:43.004325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1030/2059] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:44:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:44:43.010665 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2fc22-42ed-4d8a-8527-f621fe4041f8 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:43.021432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1030/2060] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:44:43.025052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2fc22-42ed-4d8a-8527-f621fe4041f8 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:43.029830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68138c24-7083-496b-b52f-0429b8587891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:43.030511 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68138c24-7083-496b-b52f-0429b8587891 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:43.030781 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68138c24-7083-496b-b52f-0429b8587891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:43.031101 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68138c24-7083-496b-b52f-0429b8587891 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:43.031880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68138c24-7083-496b-b52f-0429b8587891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:43.034550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68138c24-7083-496b-b52f-0429b8587891 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:43.046556 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:43.074171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1031/2061] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:43 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:43.075373 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858e962-0165-4d23-b8d2-d8d180ce71d4 service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:43.087590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858e962-0165-4d23-b8d2-d8d180ce71d4 service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:43.093375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:43.160426 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d886fd0-96e7-43d3-bd3f-fab9b8f6fc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:43.161198 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d886fd0-96e7-43d3-bd3f-fab9b8f6fc32 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:43.161496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d886fd0-96e7-43d3-bd3f-fab9b8f6fc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:43.161771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d886fd0-96e7-43d3-bd3f-fab9b8f6fc32 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:43.162531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d886fd0-96e7-43d3-bd3f-fab9b8f6fc32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:43.165345 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d886fd0-96e7-43d3-bd3f-fab9b8f6fc32 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:43.202912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1031/2062] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:43 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:43.204397 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68138c24-7083-496b-b52f-0429b8587891 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:43.216250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68138c24-7083-496b-b52f-0429b8587891 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:43.221463 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6554909-0663-4e95-97f3-7fefba16d1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:43.222217 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6554909-0663-4e95-97f3-7fefba16d1ba admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:43.222708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6554909-0663-4e95-97f3-7fefba16d1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:43.223089 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6554909-0663-4e95-97f3-7fefba16d1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:43.223969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6554909-0663-4e95-97f3-7fefba16d1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:43.225701 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6554909-0663-4e95-97f3-7fefba16d1ba admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:43.239253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1032/2063] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:43.246944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d886fd0-96e7-43d3-bd3f-fab9b8f6fc32 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:43.264795 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d886fd0-96e7-43d3-bd3f-fab9b8f6fc32 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:43.269973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf6558-ff2f-4b5f-8e81-1b22795fe122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:43.270730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf6558-ff2f-4b5f-8e81-1b22795fe122 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:43.271205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf6558-ff2f-4b5f-8e81-1b22795fe122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:43.271589 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf6558-ff2f-4b5f-8e81-1b22795fe122 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:43.272431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf6558-ff2f-4b5f-8e81-1b22795fe122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:43.278458 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf6558-ff2f-4b5f-8e81-1b22795fe122 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:43.294666 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1032/2064] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:44:43.295366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6554909-0663-4e95-97f3-7fefba16d1ba admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:43.309512 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6554909-0663-4e95-97f3-7fefba16d1ba admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:43.314743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d3a76-0b68-4bbb-aae6-e446a260ecfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:43.318103 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d3a76-0b68-4bbb-aae6-e446a260ecfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:43.318394 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d3a76-0b68-4bbb-aae6-e446a260ecfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:43.318678 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d3a76-0b68-4bbb-aae6-e446a260ecfc admin admin] PATH_INFO: `/v3/projects/4e9f7f5d606447c98583e3b93a6e6458/users/931b0a44bc904a2391b2c2181d0ca025/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:43.337001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d3a76-0b68-4bbb-aae6-e446a260ecfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e9f7f5d606447c98583e3b93a6e6458, user_id=931b0a44bc904a2391b2c2181d0ca025, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:43.339899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d3a76-0b68-4bbb-aae6-e446a260ecfc admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:43.375104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1033/2065] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:43 2022] PUT /identity/v3/projects/4e9f7f5d606447c98583e3b93a6e6458/users/931b0a44bc904a2391b2c2181d0ca025/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:43.385621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f9044-aec1-4c9d-85db-a5917ccfd04d None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:43.386008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f9044-aec1-4c9d-85db-a5917ccfd04d None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:43.386225 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f9044-aec1-4c9d-85db-a5917ccfd04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:43.444344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-d69f9044-aec1-4c9d-85db-a5917ccfd04d None None] MFA Rules not processed for user `931b0a44bc904a2391b2c2181d0ca025`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:43.449580 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d69f9044-aec1-4c9d-85db-a5917ccfd04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:43.564221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d3a76-0b68-4bbb-aae6-e446a260ecfc admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:43.566487 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d77d3a76-0b68-4bbb-aae6-e446a260ecfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:43.575154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1033/2066] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:43.576733 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f9044-aec1-4c9d-85db-a5917ccfd04d None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:43.635371 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d3a76-0b68-4bbb-aae6-e446a260ecfc admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:43.640125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d85ec9-817b-4ca4-a9b5-bbc93e0650e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:43.641044 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d85ec9-817b-4ca4-a9b5-bbc93e0650e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:43.641325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d85ec9-817b-4ca4-a9b5-bbc93e0650e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:43.641599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d85ec9-817b-4ca4-a9b5-bbc93e0650e4 admin admin] PATH_INFO: `/v3/users/2ac9950281cd45439311905a03527f57` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:43.642833 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d85ec9-817b-4ca4-a9b5-bbc93e0650e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ac9950281cd45439311905a03527f57)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:43.645322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d85ec9-817b-4ca4-a9b5-bbc93e0650e4 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:43.655634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f9044-aec1-4c9d-85db-a5917ccfd04d None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:43.664262 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9803e-9cf2-4663-980a-323db7480630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:43.665002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9803e-9cf2-4663-980a-323db7480630 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:43.668132 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9803e-9cf2-4663-980a-323db7480630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:43.668132 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9803e-9cf2-4663-980a-323db7480630 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:43.668564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9803e-9cf2-4663-980a-323db7480630 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:43.670698 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9803e-9cf2-4663-980a-323db7480630 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:43.703115 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1034/2067] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:43 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:44:43.705718 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9803e-9cf2-4663-980a-323db7480630 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:43.705869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-56d85ec9-817b-4ca4-a9b5-bbc93e0650e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ac9950281cd45439311905a03527f57'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:43.710237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:43.718560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-56d85ec9-817b-4ca4-a9b5-bbc93e0650e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ac9950281cd45439311905a03527f57'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:43.726033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-56d85ec9-817b-4ca4-a9b5-bbc93e0650e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ac9950281cd45439311905a03527f57'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:43.726773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-56d85ec9-817b-4ca4-a9b5-bbc93e0650e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ac9950281cd45439311905a03527f57'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:43.727437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-56d85ec9-817b-4ca4-a9b5-bbc93e0650e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ac9950281cd45439311905a03527f57'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:43.745200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1034/2068] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:43 2022] DELETE /identity/v3/users/2ac9950281cd45439311905a03527f57 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:43.750299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cbc36-1051-41dd-96e4-981b81a97541 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:43.750902 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cbc36-1051-41dd-96e4-981b81a97541 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:43.751327 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cbc36-1051-41dd-96e4-981b81a97541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:43.771456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-4b1cbc36-1051-41dd-96e4-981b81a97541 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:43.773294 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4b1cbc36-1051-41dd-96e4-981b81a97541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:43.790501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9803e-9cf2-4663-980a-323db7480630 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:43.804245 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:43.836557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1035/2069] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:43.846758 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cbc36-1051-41dd-96e4-981b81a97541 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:43.848571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4b1cbc36-1051-41dd-96e4-981b81a97541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:43.856121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db68cb-5b70-41ae-a6d6-eea73a019407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:43.856874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db68cb-5b70-41ae-a6d6-eea73a019407 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:43.857157 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db68cb-5b70-41ae-a6d6-eea73a019407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:43.857431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66db68cb-5b70-41ae-a6d6-eea73a019407 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:43.858170 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db68cb-5b70-41ae-a6d6-eea73a019407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:43.860172 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66db68cb-5b70-41ae-a6d6-eea73a019407 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:43.885437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1035/2070] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:43 2022] POST /identity/v3/users => generated 300 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:44:43.889430 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db68cb-5b70-41ae-a6d6-eea73a019407 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:43.891207 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-66db68cb-5b70-41ae-a6d6-eea73a019407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:43.903232 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66db68cb-5b70-41ae-a6d6-eea73a019407 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:43.908935 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d626d16-c8a9-404f-9f26-8917803f142b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:43.909671 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d626d16-c8a9-404f-9f26-8917803f142b admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:43.910372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d626d16-c8a9-404f-9f26-8917803f142b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:43.910715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d626d16-c8a9-404f-9f26-8917803f142b admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:43.911554 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d626d16-c8a9-404f-9f26-8917803f142b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:43.913215 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d626d16-c8a9-404f-9f26-8917803f142b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:43.917329 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cbc36-1051-41dd-96e4-981b81a97541 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:43.921414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1a5b1-3c83-4681-8622-91d2d8960c72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:43.922181 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1a5b1-3c83-4681-8622-91d2d8960c72 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:43.922537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1a5b1-3c83-4681-8622-91d2d8960c72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:43.922892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1a5b1-3c83-4681-8622-91d2d8960c72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:43.927318 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1036/2071] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:43.928461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d626d16-c8a9-404f-9f26-8917803f142b admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:43.933680 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1a5b1-3c83-4681-8622-91d2d8960c72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:43.935168 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1a5b1-3c83-4681-8622-91d2d8960c72 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:43.938630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d626d16-c8a9-404f-9f26-8917803f142b admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:43.950075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5eecc-8d1d-48e1-b353-7e47af86a29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:43.950075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5eecc-8d1d-48e1-b353-7e47af86a29a admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:43.950075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5eecc-8d1d-48e1-b353-7e47af86a29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:43.950075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5eecc-8d1d-48e1-b353-7e47af86a29a admin admin] PATH_INFO: `/v3/projects/5c67edfbce42495fa6fd7ffe8da1beb0/users/0a52e7d1c6094ab5b6f40d067f9b773a/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:43.950460 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1036/2072] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:44:43.969979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5eecc-8d1d-48e1-b353-7e47af86a29a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c67edfbce42495fa6fd7ffe8da1beb0, user_id=0a52e7d1c6094ab5b6f40d067f9b773a, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:43.973466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5eecc-8d1d-48e1-b353-7e47af86a29a admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:43.991697 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1037/2073] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:43 2022] PUT /identity/v3/projects/5c67edfbce42495fa6fd7ffe8da1beb0/users/0a52e7d1c6094ab5b6f40d067f9b773a/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:43.996297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1a5b1-3c83-4681-8622-91d2d8960c72 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:44.013043 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1a5b1-3c83-4681-8622-91d2d8960c72 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:44.074017 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c9eff1-3dfd-4296-9b47-a8f6d49d712a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:44.074714 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9eff1-3dfd-4296-9b47-a8f6d49d712a admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:44.074988 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9eff1-3dfd-4296-9b47-a8f6d49d712a admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:44.075327 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c9eff1-3dfd-4296-9b47-a8f6d49d712a admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:44.076036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c9eff1-3dfd-4296-9b47-a8f6d49d712a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:44.077666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c9eff1-3dfd-4296-9b47-a8f6d49d712a admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:44.091444 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1037/2074] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:44:44.092643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5eecc-8d1d-48e1-b353-7e47af86a29a admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:44.103165 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5eecc-8d1d-48e1-b353-7e47af86a29a admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:44.117887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d69d43-b30d-4d07-aa8f-bfd16712690c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:44.118862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d69d43-b30d-4d07-aa8f-bfd16712690c admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:44.119303 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d69d43-b30d-4d07-aa8f-bfd16712690c admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:44.119643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d69d43-b30d-4d07-aa8f-bfd16712690c admin admin] PATH_INFO: `/v3/projects/5c67edfbce42495fa6fd7ffe8da1beb0/users/0a52e7d1c6094ab5b6f40d067f9b773a/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:44.121725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d69d43-b30d-4d07-aa8f-bfd16712690c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c67edfbce42495fa6fd7ffe8da1beb0, user_id=0a52e7d1c6094ab5b6f40d067f9b773a, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:44.124506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d69d43-b30d-4d07-aa8f-bfd16712690c admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:44.135680 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1038/2075] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:44 2022] PUT /identity/v3/projects/5c67edfbce42495fa6fd7ffe8da1beb0/users/0a52e7d1c6094ab5b6f40d067f9b773a/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:44.147565 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8f483-adb7-4a6e-845e-f17193a4d267 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:44.147826 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8f483-adb7-4a6e-845e-f17193a4d267 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:44.148128 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8f483-adb7-4a6e-845e-f17193a4d267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:44.178065 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-9ef8f483-adb7-4a6e-845e-f17193a4d267 None None] MFA Rules not processed for user `0a52e7d1c6094ab5b6f40d067f9b773a`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:44.179938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9ef8f483-adb7-4a6e-845e-f17193a4d267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:44.249563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d69d43-b30d-4d07-aa8f-bfd16712690c admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:44.252764 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-15d69d43-b30d-4d07-aa8f-bfd16712690c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:44.259852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1038/2076] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:44.260409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:44.270859 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8f483-adb7-4a6e-845e-f17193a4d267 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:44.272524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9ef8f483-adb7-4a6e-845e-f17193a4d267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:44.317927 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:44.332581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d69d43-b30d-4d07-aa8f-bfd16712690c admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:44.337253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370ffa2-2c7e-45d0-9324-d7aa5118434a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:44.338122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370ffa2-2c7e-45d0-9324-d7aa5118434a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:44.338246 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8f483-adb7-4a6e-845e-f17193a4d267 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:44.338417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370ffa2-2c7e-45d0-9324-d7aa5118434a admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:44.338672 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370ffa2-2c7e-45d0-9324-d7aa5118434a admin admin] PATH_INFO: `/v3/projects/8da79944da71422c8d3eb6712d25f23b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:44.339906 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370ffa2-2c7e-45d0-9324-d7aa5118434a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8da79944da71422c8d3eb6712d25f23b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:44.341802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370ffa2-2c7e-45d0-9324-d7aa5118434a admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:44.341912 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfabb41-be7c-4738-bd54-2b2b69a157bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:44.342556 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfabb41-be7c-4738-bd54-2b2b69a157bb service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:44.342838 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfabb41-be7c-4738-bd54-2b2b69a157bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:44.343145 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfabb41-be7c-4738-bd54-2b2b69a157bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:44.352410 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfabb41-be7c-4738-bd54-2b2b69a157bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:44.354036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfabb41-be7c-4738-bd54-2b2b69a157bb service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:44.368055 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1039/2077] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:44:44.392330 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e370ffa2-2c7e-45d0-9324-d7aa5118434a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8da79944da71422c8d3eb6712d25f23b', 'request_id': 'req-e370ffa2-2c7e-45d0-9324-d7aa5118434a'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:44.399772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e370ffa2-2c7e-45d0-9324-d7aa5118434a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8da79944da71422c8d3eb6712d25f23b', 'request_id': 'req-e370ffa2-2c7e-45d0-9324-d7aa5118434a'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:44.408760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e370ffa2-2c7e-45d0-9324-d7aa5118434a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8da79944da71422c8d3eb6712d25f23b', 'request_id': 'req-e370ffa2-2c7e-45d0-9324-d7aa5118434a'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:44.420859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e370ffa2-2c7e-45d0-9324-d7aa5118434a admin admin] The token cache is being invalidate because project 8da79944da71422c8d3eb6712d25f23b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:44.421152 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e370ffa2-2c7e-45d0-9324-d7aa5118434a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:44.421813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e370ffa2-2c7e-45d0-9324-d7aa5118434a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:44.423535 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1039/2078] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:44 2022] DELETE /identity/v3/projects/8da79944da71422c8d3eb6712d25f23b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:44.428513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfabb41-be7c-4738-bd54-2b2b69a157bb service cinder] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:44.439669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfabb41-be7c-4738-bd54-2b2b69a157bb service cinder] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:44.503578 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661dfcd6-66b9-4833-b55a-88687dcd0b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:44.504322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dfcd6-66b9-4833-b55a-88687dcd0b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:44.504610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dfcd6-66b9-4833-b55a-88687dcd0b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:44.504891 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661dfcd6-66b9-4833-b55a-88687dcd0b91 admin admin] PATH_INFO: `/v3/users/79c6a4ede8c8421fa10c68a8bffa858e` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:44.506126 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661dfcd6-66b9-4833-b55a-88687dcd0b91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79c6a4ede8c8421fa10c68a8bffa858e)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:44.508116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661dfcd6-66b9-4833-b55a-88687dcd0b91 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:44.574808 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-661dfcd6-66b9-4833-b55a-88687dcd0b91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79c6a4ede8c8421fa10c68a8bffa858e'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:44.589935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-661dfcd6-66b9-4833-b55a-88687dcd0b91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79c6a4ede8c8421fa10c68a8bffa858e'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:44.598009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-661dfcd6-66b9-4833-b55a-88687dcd0b91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79c6a4ede8c8421fa10c68a8bffa858e'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:44.598756 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-661dfcd6-66b9-4833-b55a-88687dcd0b91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79c6a4ede8c8421fa10c68a8bffa858e'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:44.599409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-661dfcd6-66b9-4833-b55a-88687dcd0b91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79c6a4ede8c8421fa10c68a8bffa858e'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:44.615498 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1040/2079] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:44 2022] DELETE /identity/v3/users/79c6a4ede8c8421fa10c68a8bffa858e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:45.052762 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370ffa2-2c7e-45d0-9324-d7aa5118434a admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:45.054597 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e370ffa2-2c7e-45d0-9324-d7aa5118434a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:45.076144 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370ffa2-2c7e-45d0-9324-d7aa5118434a admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:45.138698 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a327b2-ca39-4a57-8d66-b768e5644b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:45.139653 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a327b2-ca39-4a57-8d66-b768e5644b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:45.139936 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a327b2-ca39-4a57-8d66-b768e5644b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:45.140214 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a327b2-ca39-4a57-8d66-b768e5644b6b admin admin] PATH_INFO: `/v3/users/931b0a44bc904a2391b2c2181d0ca025` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:45.141504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a327b2-ca39-4a57-8d66-b768e5644b6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=931b0a44bc904a2391b2c2181d0ca025)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:45.143512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a327b2-ca39-4a57-8d66-b768e5644b6b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:45.198617 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-47a327b2-ca39-4a57-8d66-b768e5644b6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '931b0a44bc904a2391b2c2181d0ca025'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:45.210839 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-47a327b2-ca39-4a57-8d66-b768e5644b6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '931b0a44bc904a2391b2c2181d0ca025'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:45.218895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-47a327b2-ca39-4a57-8d66-b768e5644b6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '931b0a44bc904a2391b2c2181d0ca025'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:45.219694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-47a327b2-ca39-4a57-8d66-b768e5644b6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '931b0a44bc904a2391b2c2181d0ca025'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:45.220305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-47a327b2-ca39-4a57-8d66-b768e5644b6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '931b0a44bc904a2391b2c2181d0ca025'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:45.233254 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:45.238905 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1040/2080] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:45 2022] DELETE /identity/v3/users/931b0a44bc904a2391b2c2181d0ca025 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:45.244793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08361c0-92df-4cb4-9389-583dcf8469d4 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:45.245369 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08361c0-92df-4cb4-9389-583dcf8469d4 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:45.245369 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08361c0-92df-4cb4-9389-583dcf8469d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:45.270723 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-a08361c0-92df-4cb4-9389-583dcf8469d4 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:45.272056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a08361c0-92df-4cb4-9389-583dcf8469d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:45.291750 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:45.345594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1041/2081] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:45.355787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a327b2-ca39-4a57-8d66-b768e5644b6b admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:45.357550 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-47a327b2-ca39-4a57-8d66-b768e5644b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:45.384603 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08361c0-92df-4cb4-9389-583dcf8469d4 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:45.386104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a08361c0-92df-4cb4-9389-583dcf8469d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:45.398901 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08361c0-92df-4cb4-9389-583dcf8469d4 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:45.412078 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2984a1-d79c-47a0-8c31-05092f53bebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:45.414133 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2984a1-d79c-47a0-8c31-05092f53bebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:45.414628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2984a1-d79c-47a0-8c31-05092f53bebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:45.415360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2984a1-d79c-47a0-8c31-05092f53bebb admin admin] PATH_INFO: `/v3/projects/bcb1e6e62e36473bb857396bab3d35e2` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:45.417486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2984a1-d79c-47a0-8c31-05092f53bebb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcb1e6e62e36473bb857396bab3d35e2)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:45.419591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2984a1-d79c-47a0-8c31-05092f53bebb admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:45.430568 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a327b2-ca39-4a57-8d66-b768e5644b6b admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:45.435928 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a512ca0-814c-4ed9-b81a-1d3211c03066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:45.436772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a512ca0-814c-4ed9-b81a-1d3211c03066 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:45.437157 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a512ca0-814c-4ed9-b81a-1d3211c03066 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:45.437636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a512ca0-814c-4ed9-b81a-1d3211c03066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:45.480193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a512ca0-814c-4ed9-b81a-1d3211c03066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:45.481976 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a512ca0-814c-4ed9-b81a-1d3211c03066 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:45.483054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1e2984a1-d79c-47a0-8c31-05092f53bebb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcb1e6e62e36473bb857396bab3d35e2', 'request_id': 'req-1e2984a1-d79c-47a0-8c31-05092f53bebb'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:45.489796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1e2984a1-d79c-47a0-8c31-05092f53bebb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcb1e6e62e36473bb857396bab3d35e2', 'request_id': 'req-1e2984a1-d79c-47a0-8c31-05092f53bebb'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:45.494135 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1041/2082] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:45.495729 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1e2984a1-d79c-47a0-8c31-05092f53bebb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcb1e6e62e36473bb857396bab3d35e2', 'request_id': 'req-1e2984a1-d79c-47a0-8c31-05092f53bebb'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:45.517547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1e2984a1-d79c-47a0-8c31-05092f53bebb admin admin] The token cache is being invalidate because project bcb1e6e62e36473bb857396bab3d35e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:45.518593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1e2984a1-d79c-47a0-8c31-05092f53bebb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:45.520115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1e2984a1-d79c-47a0-8c31-05092f53bebb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:45.522234 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1042/2083] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:45 2022] DELETE /identity/v3/projects/bcb1e6e62e36473bb857396bab3d35e2 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:45.528568 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5748117-3cdc-4ab9-aed2-1c360e4bad22 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:45.528568 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5748117-3cdc-4ab9-aed2-1c360e4bad22 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:45.528992 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5748117-3cdc-4ab9-aed2-1c360e4bad22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:45.552874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-c5748117-3cdc-4ab9-aed2-1c360e4bad22 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:45.554325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c5748117-3cdc-4ab9-aed2-1c360e4bad22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:45.661217 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1042/2084] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:45.672570 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a79c0-7023-4124-8693-45f3b73f46cd None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:45.672870 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a79c0-7023-4124-8693-45f3b73f46cd None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:45.673161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a79c0-7023-4124-8693-45f3b73f46cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:45.699132 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-856a79c0-7023-4124-8693-45f3b73f46cd None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:45.700505 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-856a79c0-7023-4124-8693-45f3b73f46cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:45.716810 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1043/2085] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:45.720752 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5748117-3cdc-4ab9-aed2-1c360e4bad22 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:45.732744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5748117-3cdc-4ab9-aed2-1c360e4bad22 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:45.738770 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484df10c-f7a1-42d4-ab4f-354253ab0e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:45.739627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484df10c-f7a1-42d4-ab4f-354253ab0e4c admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:45.739911 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484df10c-f7a1-42d4-ab4f-354253ab0e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:45.740178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484df10c-f7a1-42d4-ab4f-354253ab0e4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:45.740917 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484df10c-f7a1-42d4-ab4f-354253ab0e4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:44:45.741219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484df10c-f7a1-42d4-ab4f-354253ab0e4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:45.742846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484df10c-f7a1-42d4-ab4f-354253ab0e4c admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:45.763047 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1043/2086] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:44:45.764647 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a79c0-7023-4124-8693-45f3b73f46cd None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:45.774491 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a79c0-7023-4124-8693-45f3b73f46cd None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:45.779251 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af752e9-89bf-4d5d-8482-1a2ea2e94bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:45.780047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af752e9-89bf-4d5d-8482-1a2ea2e94bbe admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:45.780336 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af752e9-89bf-4d5d-8482-1a2ea2e94bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:45.780619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af752e9-89bf-4d5d-8482-1a2ea2e94bbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:45.781370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af752e9-89bf-4d5d-8482-1a2ea2e94bbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:45.783340 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af752e9-89bf-4d5d-8482-1a2ea2e94bbe admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:45.818932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1044/2087] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:45 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:45.823101 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484df10c-f7a1-42d4-ab4f-354253ab0e4c admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:45.845032 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484df10c-f7a1-42d4-ab4f-354253ab0e4c admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:45.846367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af752e9-89bf-4d5d-8482-1a2ea2e94bbe admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:45.848133 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-1af752e9-89bf-4d5d-8482-1a2ea2e94bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:45.925434 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c58f1-d039-4a4b-a7cf-708e5e47848c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:45.926313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c58f1-d039-4a4b-a7cf-708e5e47848c admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:45.926697 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c58f1-d039-4a4b-a7cf-708e5e47848c admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:45.927102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c58f1-d039-4a4b-a7cf-708e5e47848c admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:45.927931 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c58f1-d039-4a4b-a7cf-708e5e47848c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:45.930001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c58f1-d039-4a4b-a7cf-708e5e47848c admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:45.931563 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af752e9-89bf-4d5d-8482-1a2ea2e94bbe admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:45.936128 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:45.937264 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:45.937614 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:45.938011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0 admin admin] PATH_INFO: `/v3/projects/4e9f7f5d606447c98583e3b93a6e6458` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:45.939495 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e9f7f5d606447c98583e3b93a6e6458)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:45.943819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:45.963053 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1044/2088] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:45 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:44:45.964816 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c58f1-d039-4a4b-a7cf-708e5e47848c admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:45.969128 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:45.976292 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c58f1-d039-4a4b-a7cf-708e5e47848c admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:45.981878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334a4d8-bbb3-4270-b344-2ecca75a5e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:45.982625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334a4d8-bbb3-4270-b344-2ecca75a5e53 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:45.982912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334a4d8-bbb3-4270-b344-2ecca75a5e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:45.983225 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334a4d8-bbb3-4270-b344-2ecca75a5e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:45.983958 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334a4d8-bbb3-4270-b344-2ecca75a5e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:45.985592 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334a4d8-bbb3-4270-b344-2ecca75a5e53 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:46.003135 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1045/2089] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:44:46.007439 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334a4d8-bbb3-4270-b344-2ecca75a5e53 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:46.008176 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e9f7f5d606447c98583e3b93a6e6458', 'request_id': 'req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:46.014920 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e9f7f5d606447c98583e3b93a6e6458', 'request_id': 'req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:46.020717 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334a4d8-bbb3-4270-b344-2ecca75a5e53 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:46.024645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e9f7f5d606447c98583e3b93a6e6458', 'request_id': 'req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:46.026414 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:46.047766 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0 admin admin] The token cache is being invalidate because project 4e9f7f5d606447c98583e3b93a6e6458 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:46.048238 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:46.049123 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:46.055271 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1045/2090] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:45 2022] DELETE /identity/v3/projects/4e9f7f5d606447c98583e3b93a6e6458 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:46.057155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:46.059269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:46.125622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffd085-d963-4ec9-a5d6-fec2dcf4804a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:46.134431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffd085-d963-4ec9-a5d6-fec2dcf4804a admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:46.134431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffd085-d963-4ec9-a5d6-fec2dcf4804a admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:46.134431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ffd085-d963-4ec9-a5d6-fec2dcf4804a admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:46.134431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ffd085-d963-4ec9-a5d6-fec2dcf4804a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:46.134431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ffd085-d963-4ec9-a5d6-fec2dcf4804a admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:46.143866 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6e04a-57bd-44f7-8424-7f0c915bdef0 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:46.144064 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1046/2091] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:44:46.145018 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffd085-d963-4ec9-a5d6-fec2dcf4804a admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:46.146831 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-34ffd085-d963-4ec9-a5d6-fec2dcf4804a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:46.152642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b35077-b3e6-4224-9166-3864be3e2bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:46.153638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b35077-b3e6-4224-9166-3864be3e2bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:46.153923 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b35077-b3e6-4224-9166-3864be3e2bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:46.154199 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b35077-b3e6-4224-9166-3864be3e2bc8 admin admin] PATH_INFO: `/v3/users/0a52e7d1c6094ab5b6f40d067f9b773a` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:46.155515 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b35077-b3e6-4224-9166-3864be3e2bc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a52e7d1c6094ab5b6f40d067f9b773a)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:46.157850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b35077-b3e6-4224-9166-3864be3e2bc8 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:46.163120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ffd085-d963-4ec9-a5d6-fec2dcf4804a admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:46.168777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d802a5a-cac1-4887-aee1-0aa58f4f676c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:46.169754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d802a5a-cac1-4887-aee1-0aa58f4f676c admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:46.170628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d802a5a-cac1-4887-aee1-0aa58f4f676c admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:46.170628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d802a5a-cac1-4887-aee1-0aa58f4f676c admin admin] PATH_INFO: `/v3/projects/2052497ccd9a4343a388cc2d0b90425e/users/6278f41b27c24139b647b23b1f2df406/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:46.188757 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d802a5a-cac1-4887-aee1-0aa58f4f676c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2052497ccd9a4343a388cc2d0b90425e, user_id=6278f41b27c24139b647b23b1f2df406, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:46.191887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d802a5a-cac1-4887-aee1-0aa58f4f676c admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:46.211061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1047/2092] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:46 2022] PUT /identity/v3/projects/2052497ccd9a4343a388cc2d0b90425e/users/6278f41b27c24139b647b23b1f2df406/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:46.212660 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff87e0-3e3e-4418-8130-aa40a3c2be6e None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:46.212812 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-d0b35077-b3e6-4224-9166-3864be3e2bc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a52e7d1c6094ab5b6f40d067f9b773a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:46.213013 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff87e0-3e3e-4418-8130-aa40a3c2be6e None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:46.213214 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff87e0-3e3e-4418-8130-aa40a3c2be6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:46.232555 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-81ff87e0-3e3e-4418-8130-aa40a3c2be6e None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:46.232977 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-d0b35077-b3e6-4224-9166-3864be3e2bc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a52e7d1c6094ab5b6f40d067f9b773a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:46.233795 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-81ff87e0-3e3e-4418-8130-aa40a3c2be6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:46.243959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-d0b35077-b3e6-4224-9166-3864be3e2bc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a52e7d1c6094ab5b6f40d067f9b773a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:46.244960 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-d0b35077-b3e6-4224-9166-3864be3e2bc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a52e7d1c6094ab5b6f40d067f9b773a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:46.245989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-d0b35077-b3e6-4224-9166-3864be3e2bc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a52e7d1c6094ab5b6f40d067f9b773a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:46.263719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1046/2093] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:46 2022] DELETE /identity/v3/users/0a52e7d1c6094ab5b6f40d067f9b773a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:46.310197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1048/2094] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:46.327052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b35077-b3e6-4224-9166-3864be3e2bc8 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:46.327052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d0b35077-b3e6-4224-9166-3864be3e2bc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:46.364128 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:46.395722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b35077-b3e6-4224-9166-3864be3e2bc8 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:46.399718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a41080-9625-4700-978a-f13307561fad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:46.400478 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a41080-9625-4700-978a-f13307561fad service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:46.400834 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a41080-9625-4700-978a-f13307561fad service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:46.401178 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a41080-9625-4700-978a-f13307561fad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:46.478333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a41080-9625-4700-978a-f13307561fad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:46.479880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a41080-9625-4700-978a-f13307561fad service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:46.491854 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1047/2095] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:46.680809 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff87e0-3e3e-4418-8130-aa40a3c2be6e None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:46.683459 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-81ff87e0-3e3e-4418-8130-aa40a3c2be6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:46.698978 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff87e0-3e3e-4418-8130-aa40a3c2be6e None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:46.715576 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef5ba5-5e06-4242-a0fa-7e09eba3c333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:46.716895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef5ba5-5e06-4242-a0fa-7e09eba3c333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:46.717199 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef5ba5-5e06-4242-a0fa-7e09eba3c333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:46.717478 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef5ba5-5e06-4242-a0fa-7e09eba3c333 admin admin] PATH_INFO: `/v3/projects/5c67edfbce42495fa6fd7ffe8da1beb0` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:46.732884 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef5ba5-5e06-4242-a0fa-7e09eba3c333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c67edfbce42495fa6fd7ffe8da1beb0)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:46.748253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef5ba5-5e06-4242-a0fa-7e09eba3c333 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:46.832228 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-9cef5ba5-5e06-4242-a0fa-7e09eba3c333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c67edfbce42495fa6fd7ffe8da1beb0', 'request_id': 'req-9cef5ba5-5e06-4242-a0fa-7e09eba3c333'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:46.838456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-9cef5ba5-5e06-4242-a0fa-7e09eba3c333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c67edfbce42495fa6fd7ffe8da1beb0', 'request_id': 'req-9cef5ba5-5e06-4242-a0fa-7e09eba3c333'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:46.848197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-9cef5ba5-5e06-4242-a0fa-7e09eba3c333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c67edfbce42495fa6fd7ffe8da1beb0', 'request_id': 'req-9cef5ba5-5e06-4242-a0fa-7e09eba3c333'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:46.864884 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-9cef5ba5-5e06-4242-a0fa-7e09eba3c333 admin admin] The token cache is being invalidate because project 5c67edfbce42495fa6fd7ffe8da1beb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:46.865191 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-9cef5ba5-5e06-4242-a0fa-7e09eba3c333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:46.865828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-9cef5ba5-5e06-4242-a0fa-7e09eba3c333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:46.868221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1049/2096] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:46 2022] DELETE /identity/v3/projects/5c67edfbce42495fa6fd7ffe8da1beb0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:46.873261 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba574e5a-5990-4075-9924-de6c20548142 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:46.873625 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba574e5a-5990-4075-9924-de6c20548142 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:46.873974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba574e5a-5990-4075-9924-de6c20548142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:46.893295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-ba574e5a-5990-4075-9924-de6c20548142 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:46.894808 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ba574e5a-5990-4075-9924-de6c20548142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:46.998226 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1048/2097] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:44:47.002276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ec00f-cb02-41f2-8d4f-1ab861e48047 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:47.002572 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ec00f-cb02-41f2-8d4f-1ab861e48047 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:47.005393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ec00f-cb02-41f2-8d4f-1ab861e48047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:47.013253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:47.031290 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-d74ec00f-cb02-41f2-8d4f-1ab861e48047 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:47.032158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d74ec00f-cb02-41f2-8d4f-1ab861e48047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:47.045587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1050/2098] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:47.046622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba574e5a-5990-4075-9924-de6c20548142 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:47.062018 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba574e5a-5990-4075-9924-de6c20548142 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:47.068754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb3e69-adf4-4699-9482-53598ae52f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:47.069546 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb3e69-adf4-4699-9482-53598ae52f51 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:47.069835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb3e69-adf4-4699-9482-53598ae52f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:47.070269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb3e69-adf4-4699-9482-53598ae52f51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:47.071127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb3e69-adf4-4699-9482-53598ae52f51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:44:47.071446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb3e69-adf4-4699-9482-53598ae52f51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:47.074534 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb3e69-adf4-4699-9482-53598ae52f51 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:47.102211 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1049/2099] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:44:47.102637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ec00f-cb02-41f2-8d4f-1ab861e48047 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:47.114810 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ec00f-cb02-41f2-8d4f-1ab861e48047 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:47.129741 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009386c0-3109-40ed-94de-3afdc58870f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:47.130366 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009386c0-3109-40ed-94de-3afdc58870f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:47.130691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009386c0-3109-40ed-94de-3afdc58870f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:47.131102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009386c0-3109-40ed-94de-3afdc58870f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:47.132026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009386c0-3109-40ed-94de-3afdc58870f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:47.136670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009386c0-3109-40ed-94de-3afdc58870f1 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:47.176217 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1051/2100] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:47 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:44:47.177805 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb3e69-adf4-4699-9482-53598ae52f51 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:47.193824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb3e69-adf4-4699-9482-53598ae52f51 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:47.267346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a446c51-acd6-4c99-96bc-d5a757c93368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:47.268259 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a446c51-acd6-4c99-96bc-d5a757c93368 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:47.268552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a446c51-acd6-4c99-96bc-d5a757c93368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:47.268829 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a446c51-acd6-4c99-96bc-d5a757c93368 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:47.270191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a446c51-acd6-4c99-96bc-d5a757c93368 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:47.272385 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a446c51-acd6-4c99-96bc-d5a757c93368 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:47.283463 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:47.308005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1050/2101] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:47 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:47.309244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009386c0-3109-40ed-94de-3afdc58870f1 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:47.320931 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009386c0-3109-40ed-94de-3afdc58870f1 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:47.331588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a14983-d1b6-4e35-b3ee-6659fbc50877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:47.332280 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a14983-d1b6-4e35-b3ee-6659fbc50877 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:47.332715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a14983-d1b6-4e35-b3ee-6659fbc50877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:47.333030 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a14983-d1b6-4e35-b3ee-6659fbc50877 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:47.333965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a14983-d1b6-4e35-b3ee-6659fbc50877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:47.335836 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a14983-d1b6-4e35-b3ee-6659fbc50877 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:47.352113 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1052/2102] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:47.353730 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a446c51-acd6-4c99-96bc-d5a757c93368 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:47.370672 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a446c51-acd6-4c99-96bc-d5a757c93368 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:47.375749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb13df0-70ab-493b-9d07-d81469c150f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:47.376428 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb13df0-70ab-493b-9d07-d81469c150f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:47.376712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb13df0-70ab-493b-9d07-d81469c150f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:47.377060 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb13df0-70ab-493b-9d07-d81469c150f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:47.378078 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb13df0-70ab-493b-9d07-d81469c150f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:47.379451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb13df0-70ab-493b-9d07-d81469c150f0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:47.393897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1051/2103] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:44:47.395234 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a14983-d1b6-4e35-b3ee-6659fbc50877 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:47.409239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a14983-d1b6-4e35-b3ee-6659fbc50877 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:47.424080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa425b27-0742-4a23-8cd5-cddbf8aaacd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:47.425629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa425b27-0742-4a23-8cd5-cddbf8aaacd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:47.426163 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa425b27-0742-4a23-8cd5-cddbf8aaacd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:47.426419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa425b27-0742-4a23-8cd5-cddbf8aaacd4 admin admin] PATH_INFO: `/v3/projects/502468ce2c6543c6bf1f630d874c6f22/users/3a8bd86a47a7435c9b9b6fcc2cc9e0a8/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:47.445651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef7afa-f620-408f-bc5e-06ea4c54a032 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:47.446035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa425b27-0742-4a23-8cd5-cddbf8aaacd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=502468ce2c6543c6bf1f630d874c6f22, user_id=3a8bd86a47a7435c9b9b6fcc2cc9e0a8, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:47.446841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef7afa-f620-408f-bc5e-06ea4c54a032 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:47.447347 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef7afa-f620-408f-bc5e-06ea4c54a032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:47.449442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa425b27-0742-4a23-8cd5-cddbf8aaacd4 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:47.484057 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1053/2104] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:47 2022] PUT /identity/v3/projects/502468ce2c6543c6bf1f630d874c6f22/users/3a8bd86a47a7435c9b9b6fcc2cc9e0a8/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:47.486501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67fdbf-1c7c-474a-b375-d26d909f9e9b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:47.487739 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67fdbf-1c7c-474a-b375-d26d909f9e9b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:47.488117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67fdbf-1c7c-474a-b375-d26d909f9e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:47.493558 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-76ef7afa-f620-408f-bc5e-06ea4c54a032 None None] MFA Rules not processed for user `6278f41b27c24139b647b23b1f2df406`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:47.494796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-76ef7afa-f620-408f-bc5e-06ea4c54a032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:47.533700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-2b67fdbf-1c7c-474a-b375-d26d909f9e9b None None] MFA Rules not processed for user `3a8bd86a47a7435c9b9b6fcc2cc9e0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:47.537068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2b67fdbf-1c7c-474a-b375-d26d909f9e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:47.597920 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1052/2105] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:44:47.604356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef7afa-f620-408f-bc5e-06ea4c54a032 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:47.606464 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-76ef7afa-f620-408f-bc5e-06ea4c54a032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:47.627789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1054/2106] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:47.632154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67fdbf-1c7c-474a-b375-d26d909f9e9b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:47.712172 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef7afa-f620-408f-bc5e-06ea4c54a032 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:47.717370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61973a6f-be47-4af2-bf85-3ba1bbd72ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:47.718031 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61973a6f-be47-4af2-bf85-3ba1bbd72ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:47.718579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61973a6f-be47-4af2-bf85-3ba1bbd72ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:47.718989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61973a6f-be47-4af2-bf85-3ba1bbd72ff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:47.719755 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61973a6f-be47-4af2-bf85-3ba1bbd72ff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:47.721949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61973a6f-be47-4af2-bf85-3ba1bbd72ff0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:47.731277 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67fdbf-1c7c-474a-b375-d26d909f9e9b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:47.736800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36afd9-ddfe-40b9-8598-dd75f6976d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:47.737532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36afd9-ddfe-40b9-8598-dd75f6976d7e admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:47.737838 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36afd9-ddfe-40b9-8598-dd75f6976d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:47.738089 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36afd9-ddfe-40b9-8598-dd75f6976d7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:47.738973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36afd9-ddfe-40b9-8598-dd75f6976d7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:47.741055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36afd9-ddfe-40b9-8598-dd75f6976d7e admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:47.758226 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1053/2107] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:47 2022] POST /identity/v3/projects => generated 371 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:47.768049 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61973a6f-be47-4af2-bf85-3ba1bbd72ff0 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:47.772843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1055/2108] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:47 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:47.777243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36afd9-ddfe-40b9-8598-dd75f6976d7e admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:47.813663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:47.845040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:47.854545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61973a6f-be47-4af2-bf85-3ba1bbd72ff0 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:47.874717 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36afd9-ddfe-40b9-8598-dd75f6976d7e admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:47.880933 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39483bb8-e054-45bb-9766-6f3eb99683bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:47.880933 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39483bb8-e054-45bb-9766-6f3eb99683bf admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:47.881217 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39483bb8-e054-45bb-9766-6f3eb99683bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:47.881451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39483bb8-e054-45bb-9766-6f3eb99683bf admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:47.882156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39483bb8-e054-45bb-9766-6f3eb99683bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:47.884484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39483bb8-e054-45bb-9766-6f3eb99683bf admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:47.918085 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1056/2109] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:47 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:44:47.923805 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39483bb8-e054-45bb-9766-6f3eb99683bf admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:47.938631 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39483bb8-e054-45bb-9766-6f3eb99683bf admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:47.942678 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1eafc-b629-4d79-aba3-5fd191ebdbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:47.943859 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1eafc-b629-4d79-aba3-5fd191ebdbc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:47.944045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4830c-62c1-403b-a06b-6193139b9e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:47.944261 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1eafc-b629-4d79-aba3-5fd191ebdbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:47.944866 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4830c-62c1-403b-a06b-6193139b9e91 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:47.944988 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1eafc-b629-4d79-aba3-5fd191ebdbc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:47.945280 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4830c-62c1-403b-a06b-6193139b9e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:47.945630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4830c-62c1-403b-a06b-6193139b9e91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:47.945965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1eafc-b629-4d79-aba3-5fd191ebdbc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:47.946659 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4830c-62c1-403b-a06b-6193139b9e91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:47.948218 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1eafc-b629-4d79-aba3-5fd191ebdbc6 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:47.948457 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4830c-62c1-403b-a06b-6193139b9e91 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:47.966030 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1057/2110] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:47.967205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4830c-62c1-403b-a06b-6193139b9e91 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:47.979827 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4830c-62c1-403b-a06b-6193139b9e91 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:47.985970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f1739-8285-490a-a312-12689a16dd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:47.985970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f1739-8285-490a-a312-12689a16dd2f admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:47.985970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f1739-8285-490a-a312-12689a16dd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:47.985970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f1739-8285-490a-a312-12689a16dd2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:47.986597 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f1739-8285-490a-a312-12689a16dd2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:47.987566 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1054/2111] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:47 2022] POST /identity/v3/users => generated 292 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:47.988257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f1739-8285-490a-a312-12689a16dd2f admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:47.988947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1eafc-b629-4d79-aba3-5fd191ebdbc6 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:48.001459 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1eafc-b629-4d79-aba3-5fd191ebdbc6 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:48.002035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1058/2112] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:44:48.007235 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f1739-8285-490a-a312-12689a16dd2f admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:48.014603 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909f229-6f2f-41c1-a89a-f721f12f32ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:48.015487 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909f229-6f2f-41c1-a89a-f721f12f32ab admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:48.015980 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909f229-6f2f-41c1-a89a-f721f12f32ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:48.016560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909f229-6f2f-41c1-a89a-f721f12f32ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:48.017369 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4909f229-6f2f-41c1-a89a-f721f12f32ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:48.019073 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f1739-8285-490a-a312-12689a16dd2f admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:48.019256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4909f229-6f2f-41c1-a89a-f721f12f32ab admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:48.024976 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669edb8-a8eb-466d-a50f-14a838a5edab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:48.026672 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669edb8-a8eb-466d-a50f-14a838a5edab admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:48.027137 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669edb8-a8eb-466d-a50f-14a838a5edab admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:48.027397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669edb8-a8eb-466d-a50f-14a838a5edab admin admin] PATH_INFO: `/v3/projects/705d108de44b4b3f80a52b02f01dbf2d/users/222c4c6e0b6f4371a0d814339a50d25d/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:48.031438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1055/2113] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:48.036349 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909f229-6f2f-41c1-a89a-f721f12f32ab admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:48.046987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669edb8-a8eb-466d-a50f-14a838a5edab admin admin] RBAC: Authorizing `identity:create_grant(project_id=705d108de44b4b3f80a52b02f01dbf2d, user_id=222c4c6e0b6f4371a0d814339a50d25d, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:48.047762 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909f229-6f2f-41c1-a89a-f721f12f32ab admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:48.049947 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669edb8-a8eb-466d-a50f-14a838a5edab admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:48.055298 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7606ec-bf91-4d24-8a28-f11d2f214bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:48.055298 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7606ec-bf91-4d24-8a28-f11d2f214bad admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:48.055298 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7606ec-bf91-4d24-8a28-f11d2f214bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:48.055298 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7606ec-bf91-4d24-8a28-f11d2f214bad admin admin] PATH_INFO: `/v3/projects/4388527b12654334993397036bbb6f69/users/66a2d6b32e5945a188de043ea7fa73cf/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:48.071589 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1059/2114] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:48 2022] PUT /identity/v3/projects/705d108de44b4b3f80a52b02f01dbf2d/users/222c4c6e0b6f4371a0d814339a50d25d/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:48.083305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81ec30-7393-4013-9bc4-2842779a8a1f None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:48.083509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7606ec-bf91-4d24-8a28-f11d2f214bad admin admin] RBAC: Authorizing `identity:create_grant(project_id=4388527b12654334993397036bbb6f69, user_id=66a2d6b32e5945a188de043ea7fa73cf, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:48.083634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81ec30-7393-4013-9bc4-2842779a8a1f None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:48.083843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc81ec30-7393-4013-9bc4-2842779a8a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:48.086243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7606ec-bf91-4d24-8a28-f11d2f214bad admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:48.111109 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1056/2115] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:48 2022] PUT /identity/v3/projects/4388527b12654334993397036bbb6f69/users/66a2d6b32e5945a188de043ea7fa73cf/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:48.112871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7606ec-bf91-4d24-8a28-f11d2f214bad admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:48.116716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-cc81ec30-7393-4013-9bc4-2842779a8a1f None None] MFA Rules not processed for user `222c4c6e0b6f4371a0d814339a50d25d`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:48.117627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-cc81ec30-7393-4013-9bc4-2842779a8a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:48.196816 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7606ec-bf91-4d24-8a28-f11d2f214bad admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:48.201650 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1060/2116] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:44:48.205550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09231a6-440c-4443-97dc-dc475e021431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:48.206155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09231a6-440c-4443-97dc-dc475e021431 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:48.206453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09231a6-440c-4443-97dc-dc475e021431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:48.206741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09231a6-440c-4443-97dc-dc475e021431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:48.207398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81ec30-7393-4013-9bc4-2842779a8a1f None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:48.207807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09231a6-440c-4443-97dc-dc475e021431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:48.209522 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09231a6-440c-4443-97dc-dc475e021431 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:48.231505 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc81ec30-7393-4013-9bc4-2842779a8a1f None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:48.231637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1057/2117] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:44:48.231740 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09231a6-440c-4443-97dc-dc475e021431 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:48.232218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec50bc-b8fa-4417-a31d-8552468e1baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:48.232906 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec50bc-b8fa-4417-a31d-8552468e1baa admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:48.233195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec50bc-b8fa-4417-a31d-8552468e1baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:48.233494 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec50bc-b8fa-4417-a31d-8552468e1baa admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:48.234200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec50bc-b8fa-4417-a31d-8552468e1baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:48.236465 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec50bc-b8fa-4417-a31d-8552468e1baa admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:48.245204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09231a6-440c-4443-97dc-dc475e021431 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:48.253199 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56f360-5d84-48bb-a458-56cf05205029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:48.254323 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56f360-5d84-48bb-a458-56cf05205029 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:48.254666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56f360-5d84-48bb-a458-56cf05205029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:48.254925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56f360-5d84-48bb-a458-56cf05205029 admin admin] PATH_INFO: `/v3/projects/4388527b12654334993397036bbb6f69/users/66a2d6b32e5945a188de043ea7fa73cf/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:48.257369 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56f360-5d84-48bb-a458-56cf05205029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4388527b12654334993397036bbb6f69, user_id=66a2d6b32e5945a188de043ea7fa73cf, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:48.260133 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56f360-5d84-48bb-a458-56cf05205029 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:48.270233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1061/2118] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:48 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:48.272003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec50bc-b8fa-4417-a31d-8552468e1baa admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:48.273957 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1058/2119] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:48 2022] PUT /identity/v3/projects/4388527b12654334993397036bbb6f69/users/66a2d6b32e5945a188de043ea7fa73cf/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:48.306961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:48.341213 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec50bc-b8fa-4417-a31d-8552468e1baa admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:48.400497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559791b-98cc-486b-b706-a49f9c0aa5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:48.401121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559791b-98cc-486b-b706-a49f9c0aa5f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:48.401403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559791b-98cc-486b-b706-a49f9c0aa5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:48.401798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559791b-98cc-486b-b706-a49f9c0aa5f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:48.402399 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559791b-98cc-486b-b706-a49f9c0aa5f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:48.404493 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559791b-98cc-486b-b706-a49f9c0aa5f9 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:48.429048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1062/2120] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:48 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:48.433173 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56f360-5d84-48bb-a458-56cf05205029 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:48.445889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56f360-5d84-48bb-a458-56cf05205029 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:48.451222 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b195ce6-7f82-46d3-aecc-f3bb4d614cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:48.451920 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b195ce6-7f82-46d3-aecc-f3bb4d614cbc admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:48.452213 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b195ce6-7f82-46d3-aecc-f3bb4d614cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:48.452505 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b195ce6-7f82-46d3-aecc-f3bb4d614cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:48.453335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b195ce6-7f82-46d3-aecc-f3bb4d614cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:48.454985 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b195ce6-7f82-46d3-aecc-f3bb4d614cbc admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:48.467058 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1059/2121] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:48.471858 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559791b-98cc-486b-b706-a49f9c0aa5f9 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:48.483872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559791b-98cc-486b-b706-a49f9c0aa5f9 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:48.488839 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899cb39-fb49-42e3-96ca-04a4ec8da320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:48.489799 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899cb39-fb49-42e3-96ca-04a4ec8da320 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:48.489997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899cb39-fb49-42e3-96ca-04a4ec8da320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:48.490299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899cb39-fb49-42e3-96ca-04a4ec8da320 admin admin] PATH_INFO: `/v3/projects/061fd45b758d44fabde4571a13571210/users/e58588a7dc4d4c53a12b6bf454aa515f/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:48.506378 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899cb39-fb49-42e3-96ca-04a4ec8da320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=061fd45b758d44fabde4571a13571210, user_id=e58588a7dc4d4c53a12b6bf454aa515f, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:48.509255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899cb39-fb49-42e3-96ca-04a4ec8da320 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:48.526376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1063/2122] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:48 2022] PUT /identity/v3/projects/061fd45b758d44fabde4571a13571210/users/e58588a7dc4d4c53a12b6bf454aa515f/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:44:48.530506 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b195ce6-7f82-46d3-aecc-f3bb4d614cbc admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:48.542301 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b195ce6-7f82-46d3-aecc-f3bb4d614cbc admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:48.576424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:48.607756 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb2f64-1c3c-4a20-bd37-e7dfcefdb953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:48.608447 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb2f64-1c3c-4a20-bd37-e7dfcefdb953 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:48.608728 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb2f64-1c3c-4a20-bd37-e7dfcefdb953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:48.609002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb2f64-1c3c-4a20-bd37-e7dfcefdb953 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:48.609798 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb2f64-1c3c-4a20-bd37-e7dfcefdb953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:48.611452 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb2f64-1c3c-4a20-bd37-e7dfcefdb953 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:48.621575 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1060/2123] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:48.626316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899cb39-fb49-42e3-96ca-04a4ec8da320 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:48.641561 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899cb39-fb49-42e3-96ca-04a4ec8da320 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:48.652981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b738b-cfe4-4448-9058-d3e3065c3cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:48.654328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b738b-cfe4-4448-9058-d3e3065c3cac admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:48.654709 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b738b-cfe4-4448-9058-d3e3065c3cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:48.655167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b738b-cfe4-4448-9058-d3e3065c3cac admin admin] PATH_INFO: `/v3/projects/061fd45b758d44fabde4571a13571210/users/e58588a7dc4d4c53a12b6bf454aa515f/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:48.658488 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b738b-cfe4-4448-9058-d3e3065c3cac admin admin] RBAC: Authorizing `identity:create_grant(project_id=061fd45b758d44fabde4571a13571210, user_id=e58588a7dc4d4c53a12b6bf454aa515f, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:48.662128 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b738b-cfe4-4448-9058-d3e3065c3cac admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:48.677564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1064/2124] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:48 2022] PUT /identity/v3/projects/061fd45b758d44fabde4571a13571210/users/e58588a7dc4d4c53a12b6bf454aa515f/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:48.693122 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac52f4-486d-47e7-8b6c-bef3657fb487 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:48.693344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac52f4-486d-47e7-8b6c-bef3657fb487 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:48.693582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac52f4-486d-47e7-8b6c-bef3657fb487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:48.733516 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-8eac52f4-486d-47e7-8b6c-bef3657fb487 None None] MFA Rules not processed for user `e58588a7dc4d4c53a12b6bf454aa515f`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:48.734945 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8eac52f4-486d-47e7-8b6c-bef3657fb487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:48.812776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1061/2125] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:48 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:48.828964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b738b-cfe4-4448-9058-d3e3065c3cac admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:48.830747 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4f5b738b-cfe4-4448-9058-d3e3065c3cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:48.894057 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b738b-cfe4-4448-9058-d3e3065c3cac admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:48.897752 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e2bf0-945d-4787-8608-bfff782c6bd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:48.898411 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e2bf0-945d-4787-8608-bfff782c6bd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:48.898694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e2bf0-945d-4787-8608-bfff782c6bd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:48.898965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e2bf0-945d-4787-8608-bfff782c6bd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:48.945192 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:48.969590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e2bf0-945d-4787-8608-bfff782c6bd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:48.971117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e2bf0-945d-4787-8608-bfff782c6bd9 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:48.985651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1065/2126] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:49.322951 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757163d-d087-4370-a138-d299316c68b4 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:49.323477 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757163d-d087-4370-a138-d299316c68b4 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:49.323776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757163d-d087-4370-a138-d299316c68b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:49.359719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-f757163d-d087-4370-a138-d299316c68b4 None None] MFA Rules not processed for user `66a2d6b32e5945a188de043ea7fa73cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:49.361559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f757163d-d087-4370-a138-d299316c68b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:49.462310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1062/2127] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:49 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:49.474086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e2bf0-945d-4787-8608-bfff782c6bd9 service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:49.482533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e2bf0-945d-4787-8608-bfff782c6bd9 service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:49.486556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d5b48-a224-4d90-b033-fa7c45aae6ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:49.487307 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d5b48-a224-4d90-b033-fa7c45aae6ca service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:49.487650 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d5b48-a224-4d90-b033-fa7c45aae6ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:49.487992 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d5b48-a224-4d90-b033-fa7c45aae6ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:49.545979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d5b48-a224-4d90-b033-fa7c45aae6ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:49.547548 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d5b48-a224-4d90-b033-fa7c45aae6ca service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:49.556381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1066/2128] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:54.988363 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757163d-d087-4370-a138-d299316c68b4 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:54.989872 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f757163d-d087-4370-a138-d299316c68b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:55.002740 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757163d-d087-4370-a138-d299316c68b4 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:55.007216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aff493-1b26-45b8-94fe-387ffa105c5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:55.007869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aff493-1b26-45b8-94fe-387ffa105c5a service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:55.009681 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aff493-1b26-45b8-94fe-387ffa105c5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:55.009681 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aff493-1b26-45b8-94fe-387ffa105c5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:55.047974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:55.073522 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aff493-1b26-45b8-94fe-387ffa105c5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:55.075920 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aff493-1b26-45b8-94fe-387ffa105c5a service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:55.088464 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1063/2129] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:55.792767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d5b48-a224-4d90-b033-fa7c45aae6ca service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:55.802425 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d5b48-a224-4d90-b033-fa7c45aae6ca service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:55.806928 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c42698-24a2-4c8f-ba72-ea29f8edfce3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:55.807913 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c42698-24a2-4c8f-ba72-ea29f8edfce3 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:55.808045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c42698-24a2-4c8f-ba72-ea29f8edfce3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:55.808337 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c42698-24a2-4c8f-ba72-ea29f8edfce3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:55.828370 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c42698-24a2-4c8f-ba72-ea29f8edfce3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:55.829969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c42698-24a2-4c8f-ba72-ea29f8edfce3 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:55.840188 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1067/2130] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:55 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:55.914215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aff493-1b26-45b8-94fe-387ffa105c5a service cinder] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:55.928759 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aff493-1b26-45b8-94fe-387ffa105c5a service cinder] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:55.932724 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7e627-c10b-4c80-8a01-5213eaf0969f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:55.933541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7e627-c10b-4c80-8a01-5213eaf0969f service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:55.933888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7e627-c10b-4c80-8a01-5213eaf0969f service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:55.934287 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7e627-c10b-4c80-8a01-5213eaf0969f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:55.944262 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7e627-c10b-4c80-8a01-5213eaf0969f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:55.945858 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7e627-c10b-4c80-8a01-5213eaf0969f service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:55.956140 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1064/2131] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:57.771390 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c42698-24a2-4c8f-ba72-ea29f8edfce3 service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:57.772914 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-66c42698-24a2-4c8f-ba72-ea29f8edfce3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:57.850114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c42698-24a2-4c8f-ba72-ea29f8edfce3 service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:57.856161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3eee51-2659-4a2c-9c25-d595e6ccbe82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:57.857490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3eee51-2659-4a2c-9c25-d595e6ccbe82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:57.857921 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3eee51-2659-4a2c-9c25-d595e6ccbe82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:57.858343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3eee51-2659-4a2c-9c25-d595e6ccbe82 admin admin] PATH_INFO: `/v3/users/7d36589cc19944f7ad37c3f6fd709b08` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:57.860526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3eee51-2659-4a2c-9c25-d595e6ccbe82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d36589cc19944f7ad37c3f6fd709b08)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:57.862811 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3eee51-2659-4a2c-9c25-d595e6ccbe82 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:57.922534 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6d3eee51-2659-4a2c-9c25-d595e6ccbe82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d36589cc19944f7ad37c3f6fd709b08'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:57.934417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6d3eee51-2659-4a2c-9c25-d595e6ccbe82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d36589cc19944f7ad37c3f6fd709b08'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:57.936368 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:57.942209 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6d3eee51-2659-4a2c-9c25-d595e6ccbe82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d36589cc19944f7ad37c3f6fd709b08'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:57.943222 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6d3eee51-2659-4a2c-9c25-d595e6ccbe82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d36589cc19944f7ad37c3f6fd709b08'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:57.943933 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6d3eee51-2659-4a2c-9c25-d595e6ccbe82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d36589cc19944f7ad37c3f6fd709b08'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:57.957687 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1068/2132] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:57 2022] DELETE /identity/v3/users/7d36589cc19944f7ad37c3f6fd709b08 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:57.962300 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e5f12-3566-4345-b6e0-dd65407e4798 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:57.962800 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e5f12-3566-4345-b6e0-dd65407e4798 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:57.962900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e5f12-3566-4345-b6e0-dd65407e4798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:57.980231 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-220e5f12-3566-4345-b6e0-dd65407e4798 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:57.981420 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-220e5f12-3566-4345-b6e0-dd65407e4798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:58.039088 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1065/2133] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:58.049799 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3eee51-2659-4a2c-9c25-d595e6ccbe82 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:58.051771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6d3eee51-2659-4a2c-9c25-d595e6ccbe82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:58.123125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3eee51-2659-4a2c-9c25-d595e6ccbe82 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:58.127781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71dfc2-9e1f-40af-a7da-ef4eb34117d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:58.128443 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71dfc2-9e1f-40af-a7da-ef4eb34117d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:58.128721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71dfc2-9e1f-40af-a7da-ef4eb34117d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:58.128995 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71dfc2-9e1f-40af-a7da-ef4eb34117d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:58.138575 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71dfc2-9e1f-40af-a7da-ef4eb34117d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:58.140043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71dfc2-9e1f-40af-a7da-ef4eb34117d0 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:58.149792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1069/2134] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:44:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:44:58.431834 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e5f12-3566-4345-b6e0-dd65407e4798 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:58.433708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-220e5f12-3566-4345-b6e0-dd65407e4798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:58.449715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e5f12-3566-4345-b6e0-dd65407e4798 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:58.455842 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:58.456839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:58.457272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:58.457370 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64 admin admin] PATH_INFO: `/v3/projects/19e2fcfba7b64bb5812c82d0944f4adc` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:58.458689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19e2fcfba7b64bb5812c82d0944f4adc)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:58.460669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:58.513059 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19e2fcfba7b64bb5812c82d0944f4adc', 'request_id': 'req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:58.522405 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19e2fcfba7b64bb5812c82d0944f4adc', 'request_id': 'req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:58.530194 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19e2fcfba7b64bb5812c82d0944f4adc', 'request_id': 'req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:58.547748 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64 admin admin] The token cache is being invalidate because project 19e2fcfba7b64bb5812c82d0944f4adc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:58.548288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:58.549104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:58.551319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1066/2135] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:58 2022] DELETE /identity/v3/projects/19e2fcfba7b64bb5812c82d0944f4adc => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:58.555241 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71dfc2-9e1f-40af-a7da-ef4eb34117d0 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:58.574421 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71dfc2-9e1f-40af-a7da-ef4eb34117d0 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:58.629461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1cd31d-fe6e-4137-8a45-d0cb5cb2bdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:58.630393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cd31d-fe6e-4137-8a45-d0cb5cb2bdd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:58.630707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cd31d-fe6e-4137-8a45-d0cb5cb2bdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:58.630980 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cd31d-fe6e-4137-8a45-d0cb5cb2bdd6 admin admin] PATH_INFO: `/v3/users/19e7f57412964d75859506651a137d47` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:58.632240 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1cd31d-fe6e-4137-8a45-d0cb5cb2bdd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19e7f57412964d75859506651a137d47)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:58.634407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1cd31d-fe6e-4137-8a45-d0cb5cb2bdd6 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:58.686932 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-be1cd31d-fe6e-4137-8a45-d0cb5cb2bdd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19e7f57412964d75859506651a137d47'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:58.700096 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-be1cd31d-fe6e-4137-8a45-d0cb5cb2bdd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19e7f57412964d75859506651a137d47'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:58.707431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-be1cd31d-fe6e-4137-8a45-d0cb5cb2bdd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19e7f57412964d75859506651a137d47'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:58.708158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-be1cd31d-fe6e-4137-8a45-d0cb5cb2bdd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19e7f57412964d75859506651a137d47'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:58.708784 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-be1cd31d-fe6e-4137-8a45-d0cb5cb2bdd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19e7f57412964d75859506651a137d47'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:58.724172 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1070/2136] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:58 2022] DELETE /identity/v3/users/19e7f57412964d75859506651a137d47 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:59.014545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:59.016098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:59.030048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7d48a-ae7d-44b3-b57e-4cbe1511de64 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:59.091209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:59.107159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05ac45-f491-4f92-80f1-9637a9409be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:59.108054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05ac45-f491-4f92-80f1-9637a9409be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:59.108337 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05ac45-f491-4f92-80f1-9637a9409be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:59.109164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05ac45-f491-4f92-80f1-9637a9409be0 admin admin] PATH_INFO: `/v3/projects/f216dbbb2bce4ef9be103afb5d2d5f66` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:59.109867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05ac45-f491-4f92-80f1-9637a9409be0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f216dbbb2bce4ef9be103afb5d2d5f66)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:59.111944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05ac45-f491-4f92-80f1-9637a9409be0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:59.171557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1f05ac45-f491-4f92-80f1-9637a9409be0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f216dbbb2bce4ef9be103afb5d2d5f66', 'request_id': 'req-1f05ac45-f491-4f92-80f1-9637a9409be0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:59.177875 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1f05ac45-f491-4f92-80f1-9637a9409be0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f216dbbb2bce4ef9be103afb5d2d5f66', 'request_id': 'req-1f05ac45-f491-4f92-80f1-9637a9409be0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:59.183249 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1f05ac45-f491-4f92-80f1-9637a9409be0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f216dbbb2bce4ef9be103afb5d2d5f66', 'request_id': 'req-1f05ac45-f491-4f92-80f1-9637a9409be0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:59.196576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1f05ac45-f491-4f92-80f1-9637a9409be0 admin admin] The token cache is being invalidate because project f216dbbb2bce4ef9be103afb5d2d5f66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:44:59.196868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1f05ac45-f491-4f92-80f1-9637a9409be0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:59.197489 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1f05ac45-f491-4f92-80f1-9637a9409be0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:44:59.199100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1067/2137] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:44:59 2022] DELETE /identity/v3/projects/f216dbbb2bce4ef9be103afb5d2d5f66 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:59.204810 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58dba2d-69db-4dd8-b32c-6cfcc3757252 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:59.205388 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58dba2d-69db-4dd8-b32c-6cfcc3757252 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:59.205388 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58dba2d-69db-4dd8-b32c-6cfcc3757252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:59.217647 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:59.224522 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-c58dba2d-69db-4dd8-b32c-6cfcc3757252 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:59.226338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c58dba2d-69db-4dd8-b32c-6cfcc3757252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:59.290305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1071/2138] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:44:59.298477 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adbb6c-47f1-4309-9fb3-1c5cc2312e06 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:59.298695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adbb6c-47f1-4309-9fb3-1c5cc2312e06 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:59.299050 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adbb6c-47f1-4309-9fb3-1c5cc2312e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:59.318589 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-81adbb6c-47f1-4309-9fb3-1c5cc2312e06 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:59.319939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-81adbb6c-47f1-4309-9fb3-1c5cc2312e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:59.329571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1068/2139] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:59.334428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58dba2d-69db-4dd8-b32c-6cfcc3757252 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:59.348229 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58dba2d-69db-4dd8-b32c-6cfcc3757252 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:59.353820 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd7e04-024e-488b-a6da-e7df6bda6d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:59.354487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd7e04-024e-488b-a6da-e7df6bda6d05 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:59.354772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd7e04-024e-488b-a6da-e7df6bda6d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:59.355082 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd7e04-024e-488b-a6da-e7df6bda6d05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:59.355829 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd7e04-024e-488b-a6da-e7df6bda6d05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:44:59.356142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd7e04-024e-488b-a6da-e7df6bda6d05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:59.357774 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd7e04-024e-488b-a6da-e7df6bda6d05 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:59.370039 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1072/2140] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:44:59.374377 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81adbb6c-47f1-4309-9fb3-1c5cc2312e06 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:59.383886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81adbb6c-47f1-4309-9fb3-1c5cc2312e06 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:59.389189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5e04d-e115-484a-8ac9-80a87e3dd62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:59.389825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5e04d-e115-484a-8ac9-80a87e3dd62b admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:59.390112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5e04d-e115-484a-8ac9-80a87e3dd62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:59.390391 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5e04d-e115-484a-8ac9-80a87e3dd62b admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:59.391137 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5e04d-e115-484a-8ac9-80a87e3dd62b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:59.393061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5e04d-e115-484a-8ac9-80a87e3dd62b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:59.421224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1069/2141] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:59 2022] POST /identity/v3/projects => generated 357 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:59.425519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd7e04-024e-488b-a6da-e7df6bda6d05 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:59.436255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd7e04-024e-488b-a6da-e7df6bda6d05 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:59.494611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ddf23-fa06-4095-a68f-925fa1157127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:59.495332 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ddf23-fa06-4095-a68f-925fa1157127 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:59.495604 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ddf23-fa06-4095-a68f-925fa1157127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:59.495879 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ddf23-fa06-4095-a68f-925fa1157127 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:59.496585 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ddf23-fa06-4095-a68f-925fa1157127 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:59.498528 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ddf23-fa06-4095-a68f-925fa1157127 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:59.521681 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1073/2142] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:59 2022] POST /identity/v3/users => generated 279 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:59.525916 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5e04d-e115-484a-8ac9-80a87e3dd62b admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:59.536923 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5e04d-e115-484a-8ac9-80a87e3dd62b admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:59.541933 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b87d50-c93d-444c-bbb6-00265c7f509e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:59.542655 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b87d50-c93d-444c-bbb6-00265c7f509e admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:59.542936 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b87d50-c93d-444c-bbb6-00265c7f509e admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:59.543243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b87d50-c93d-444c-bbb6-00265c7f509e admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:59.543981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b87d50-c93d-444c-bbb6-00265c7f509e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:59.545649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b87d50-c93d-444c-bbb6-00265c7f509e admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:59.556542 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1070/2143] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:59.560561 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ddf23-fa06-4095-a68f-925fa1157127 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:59.571039 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ddf23-fa06-4095-a68f-925fa1157127 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:59.576478 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01da50-447e-48fc-aea9-3fcdac9411e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:59.576656 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01da50-447e-48fc-aea9-3fcdac9411e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:59.576941 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01da50-447e-48fc-aea9-3fcdac9411e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:59.577219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01da50-447e-48fc-aea9-3fcdac9411e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:59.577951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01da50-447e-48fc-aea9-3fcdac9411e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:59.579594 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01da50-447e-48fc-aea9-3fcdac9411e3 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:59.590400 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1074/2144] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:44:59.594746 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b87d50-c93d-444c-bbb6-00265c7f509e admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:59.605762 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b87d50-c93d-444c-bbb6-00265c7f509e admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:59.611083 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6957e-4f30-417b-843d-e31e4c53f972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:59.612005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6957e-4f30-417b-843d-e31e4c53f972 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:59.612318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6957e-4f30-417b-843d-e31e4c53f972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:59.612610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd6957e-4f30-417b-843d-e31e4c53f972 admin admin] PATH_INFO: `/v3/projects/452eab20124f4fa1aadb94173c8929f9/users/b5deb44456514ca98a43d5d49521d65d/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:59.628619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6957e-4f30-417b-843d-e31e4c53f972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=452eab20124f4fa1aadb94173c8929f9, user_id=b5deb44456514ca98a43d5d49521d65d, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:59.631333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd6957e-4f30-417b-843d-e31e4c53f972 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:59.652102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1071/2145] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:44:59 2022] PUT /identity/v3/projects/452eab20124f4fa1aadb94173c8929f9/users/b5deb44456514ca98a43d5d49521d65d/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:44:59.657963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f1029-51fd-4af6-a53d-961e4aad94a5 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:59.658592 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f1029-51fd-4af6-a53d-961e4aad94a5 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:59.659051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f1029-51fd-4af6-a53d-961e4aad94a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:59.688364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-e90f1029-51fd-4af6-a53d-961e4aad94a5 None None] MFA Rules not processed for user `b5deb44456514ca98a43d5d49521d65d`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:44:59.689592 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e90f1029-51fd-4af6-a53d-961e4aad94a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:59.758878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1075/2146] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:44:59 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:44:59.763156 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6957e-4f30-417b-843d-e31e4c53f972 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:59.834062 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd6957e-4f30-417b-843d-e31e4c53f972 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:59.838706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f63a7-5e20-4472-955e-7fd790fc0b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:59.839418 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f63a7-5e20-4472-955e-7fd790fc0b0f admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:59.839704 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f63a7-5e20-4472-955e-7fd790fc0b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:59.839984 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f63a7-5e20-4472-955e-7fd790fc0b0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:59.840696 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f63a7-5e20-4472-955e-7fd790fc0b0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:59.842664 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f63a7-5e20-4472-955e-7fd790fc0b0f admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:44:59.873608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:59.884465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1072/2147] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:44:59 2022] POST /identity/v3/projects => generated 355 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:44:59.887720 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f1029-51fd-4af6-a53d-961e4aad94a5 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:59.893845 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:44:59.901041 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f1029-51fd-4af6-a53d-961e4aad94a5 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:44:59.937802 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f63a7-5e20-4472-955e-7fd790fc0b0f admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:44:59.943579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9f3f63a7-5e20-4472-955e-7fd790fc0b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:44:59.982699 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b356521-5762-4cff-8d44-1af01836898d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:44:59.983832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b356521-5762-4cff-8d44-1af01836898d admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:44:59.984122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b356521-5762-4cff-8d44-1af01836898d admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:44:59.984968 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b356521-5762-4cff-8d44-1af01836898d admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:44:59.985374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b356521-5762-4cff-8d44-1af01836898d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:44:59.987536 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b356521-5762-4cff-8d44-1af01836898d admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:00.013035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1076/2148] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:44:59 2022] POST /identity/v3/users => generated 284 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:00.017061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b356521-5762-4cff-8d44-1af01836898d admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:00.027099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b356521-5762-4cff-8d44-1af01836898d admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:00.031936 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e83e5-97ee-449c-8666-bbbe06264c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:00.032712 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e83e5-97ee-449c-8666-bbbe06264c56 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:00.033061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e83e5-97ee-449c-8666-bbbe06264c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:00.033426 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e83e5-97ee-449c-8666-bbbe06264c56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:00.034257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386e83e5-97ee-449c-8666-bbbe06264c56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:00.036331 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386e83e5-97ee-449c-8666-bbbe06264c56 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:00.036742 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f63a7-5e20-4472-955e-7fd790fc0b0f admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:00.041531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844ba68-3d04-48f4-9b65-578bed417fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:00.042408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844ba68-3d04-48f4-9b65-578bed417fe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:00.042697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844ba68-3d04-48f4-9b65-578bed417fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:00.042967 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844ba68-3d04-48f4-9b65-578bed417fe5 admin admin] PATH_INFO: `/v3/users/3a8bd86a47a7435c9b9b6fcc2cc9e0a8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:00.044315 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844ba68-3d04-48f4-9b65-578bed417fe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a8bd86a47a7435c9b9b6fcc2cc9e0a8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:00.046264 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844ba68-3d04-48f4-9b65-578bed417fe5 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:00.047350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1077/2149] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:45:00.051729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e83e5-97ee-449c-8666-bbbe06264c56 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:00.061226 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e83e5-97ee-449c-8666-bbbe06264c56 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:00.066097 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ca67c-efff-4adf-8080-f188cfff5425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:00.066895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ca67c-efff-4adf-8080-f188cfff5425 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:00.067213 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ca67c-efff-4adf-8080-f188cfff5425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:00.067492 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ca67c-efff-4adf-8080-f188cfff5425 admin admin] PATH_INFO: `/v3/projects/a413be817ae147f9815de51a8bb7352e/users/c7cb83e0b00343b59593d38ec8b340dc/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:00.082983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ca67c-efff-4adf-8080-f188cfff5425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a413be817ae147f9815de51a8bb7352e, user_id=c7cb83e0b00343b59593d38ec8b340dc, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:00.085803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ca67c-efff-4adf-8080-f188cfff5425 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:00.098688 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e844ba68-3d04-48f4-9b65-578bed417fe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a8bd86a47a7435c9b9b6fcc2cc9e0a8'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:00.100361 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1078/2150] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:00 2022] PUT /identity/v3/projects/a413be817ae147f9815de51a8bb7352e/users/c7cb83e0b00343b59593d38ec8b340dc/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:45:00.104278 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ca67c-efff-4adf-8080-f188cfff5425 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:00.112080 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e844ba68-3d04-48f4-9b65-578bed417fe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a8bd86a47a7435c9b9b6fcc2cc9e0a8'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:00.119392 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e844ba68-3d04-48f4-9b65-578bed417fe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a8bd86a47a7435c9b9b6fcc2cc9e0a8'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:00.120170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e844ba68-3d04-48f4-9b65-578bed417fe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a8bd86a47a7435c9b9b6fcc2cc9e0a8'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:00.121032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e844ba68-3d04-48f4-9b65-578bed417fe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a8bd86a47a7435c9b9b6fcc2cc9e0a8'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:00.135657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1073/2151] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:44:59 2022] DELETE /identity/v3/users/3a8bd86a47a7435c9b9b6fcc2cc9e0a8 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:00.140565 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1626f-edef-4331-8dbb-77180e0a040a None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:00.140883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1626f-edef-4331-8dbb-77180e0a040a None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:00.141182 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1626f-edef-4331-8dbb-77180e0a040a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:00.160270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-98b1626f-edef-4331-8dbb-77180e0a040a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:00.161501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-98b1626f-edef-4331-8dbb-77180e0a040a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:00.168277 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ca67c-efff-4adf-8080-f188cfff5425 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:00.231416 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1074/2152] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:45:00.237014 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a5f42-071e-49cd-ad9f-0cd55d123067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:00.237757 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a5f42-071e-49cd-ad9f-0cd55d123067 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:00.238045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a5f42-071e-49cd-ad9f-0cd55d123067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:00.238316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a5f42-071e-49cd-ad9f-0cd55d123067 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:00.239095 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a5f42-071e-49cd-ad9f-0cd55d123067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:00.240813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a5f42-071e-49cd-ad9f-0cd55d123067 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:00.243210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1626f-edef-4331-8dbb-77180e0a040a None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:00.245346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-98b1626f-edef-4331-8dbb-77180e0a040a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:00.247027 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:00.252789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1079/2153] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:00 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:45:00.257036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a5f42-071e-49cd-ad9f-0cd55d123067 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:00.260083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-011a5f42-071e-49cd-ad9f-0cd55d123067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:00.268171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:00.274873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a5f42-071e-49cd-ad9f-0cd55d123067 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:00.280784 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643919b-6ac4-49b9-abb5-2d9c43ae81d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:00.281691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643919b-6ac4-49b9-abb5-2d9c43ae81d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:00.281978 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643919b-6ac4-49b9-abb5-2d9c43ae81d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:00.282261 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643919b-6ac4-49b9-abb5-2d9c43ae81d1 admin admin] PATH_INFO: `/v3/projects/a413be817ae147f9815de51a8bb7352e/users/c7cb83e0b00343b59593d38ec8b340dc/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:00.284343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643919b-6ac4-49b9-abb5-2d9c43ae81d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a413be817ae147f9815de51a8bb7352e, user_id=c7cb83e0b00343b59593d38ec8b340dc, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:00.287523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643919b-6ac4-49b9-abb5-2d9c43ae81d1 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:00.308913 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1080/2154] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:00 2022] PUT /identity/v3/projects/a413be817ae147f9815de51a8bb7352e/users/c7cb83e0b00343b59593d38ec8b340dc/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:00.310842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8778a3-1952-44bf-a0fc-273ae8359379 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:00.311177 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8778a3-1952-44bf-a0fc-273ae8359379 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:00.311828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8778a3-1952-44bf-a0fc-273ae8359379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:00.327329 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1626f-edef-4331-8dbb-77180e0a040a None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:00.340490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-8f8778a3-1952-44bf-a0fc-273ae8359379 None None] MFA Rules not processed for user `c7cb83e0b00343b59593d38ec8b340dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:00.341841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8f8778a3-1952-44bf-a0fc-273ae8359379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:00.379239 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc249dd8-4276-4f4a-841a-35abbdbb1592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:00.379897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc249dd8-4276-4f4a-841a-35abbdbb1592 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:00.380236 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc249dd8-4276-4f4a-841a-35abbdbb1592 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:00.380560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc249dd8-4276-4f4a-841a-35abbdbb1592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:00.413992 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1081/2155] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:00 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:00.425843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8778a3-1952-44bf-a0fc-273ae8359379 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:00.427305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8f8778a3-1952-44bf-a0fc-273ae8359379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:00.441711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc249dd8-4276-4f4a-841a-35abbdbb1592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:00.443309 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc249dd8-4276-4f4a-841a-35abbdbb1592 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:00.453100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1075/2156] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:00.491452 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8778a3-1952-44bf-a0fc-273ae8359379 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:00.495684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b81d26-f183-4343-8e94-6e8fa7980959 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:00.496354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b81d26-f183-4343-8e94-6e8fa7980959 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:00.496639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b81d26-f183-4343-8e94-6e8fa7980959 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:00.496918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b81d26-f183-4343-8e94-6e8fa7980959 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:00.507752 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b81d26-f183-4343-8e94-6e8fa7980959 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:00.509759 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b81d26-f183-4343-8e94-6e8fa7980959 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:00.526809 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1082/2157] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:00 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:45:00.604528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc249dd8-4276-4f4a-841a-35abbdbb1592 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:00.615412 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc249dd8-4276-4f4a-841a-35abbdbb1592 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:00.633116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06718b42-96f6-49b8-a14d-a2d21c4e3d43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:00.633830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06718b42-96f6-49b8-a14d-a2d21c4e3d43 service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:00.634117 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06718b42-96f6-49b8-a14d-a2d21c4e3d43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:00.634402 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06718b42-96f6-49b8-a14d-a2d21c4e3d43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:00.636226 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-06718b42-96f6-49b8-a14d-a2d21c4e3d43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:00.709743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06718b42-96f6-49b8-a14d-a2d21c4e3d43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:00.711231 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06718b42-96f6-49b8-a14d-a2d21c4e3d43 service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:00.720296 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1076/2158] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:00 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:00.755670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b81d26-f183-4343-8e94-6e8fa7980959 service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:00.757592 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f7b81d26-f183-4343-8e94-6e8fa7980959 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:00.780611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b81d26-f183-4343-8e94-6e8fa7980959 service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:00.786712 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bff43-26aa-4fbd-9278-4e855cfff04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:00.787675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bff43-26aa-4fbd-9278-4e855cfff04a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:00.787958 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bff43-26aa-4fbd-9278-4e855cfff04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:00.788421 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bff43-26aa-4fbd-9278-4e855cfff04a admin admin] PATH_INFO: `/v3/projects/502468ce2c6543c6bf1f630d874c6f22` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:00.789621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9bff43-26aa-4fbd-9278-4e855cfff04a admin admin] RBAC: Authorizing `identity:delete_project(project_id=502468ce2c6543c6bf1f630d874c6f22)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:00.791733 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9bff43-26aa-4fbd-9278-4e855cfff04a admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:00.851122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3c9bff43-26aa-4fbd-9278-4e855cfff04a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '502468ce2c6543c6bf1f630d874c6f22', 'request_id': 'req-3c9bff43-26aa-4fbd-9278-4e855cfff04a'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:00.859453 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3c9bff43-26aa-4fbd-9278-4e855cfff04a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '502468ce2c6543c6bf1f630d874c6f22', 'request_id': 'req-3c9bff43-26aa-4fbd-9278-4e855cfff04a'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:00.872193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3c9bff43-26aa-4fbd-9278-4e855cfff04a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '502468ce2c6543c6bf1f630d874c6f22', 'request_id': 'req-3c9bff43-26aa-4fbd-9278-4e855cfff04a'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:00.881091 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:00.886151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3c9bff43-26aa-4fbd-9278-4e855cfff04a admin admin] The token cache is being invalidate because project 502468ce2c6543c6bf1f630d874c6f22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:00.886434 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3c9bff43-26aa-4fbd-9278-4e855cfff04a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:00.887428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3c9bff43-26aa-4fbd-9278-4e855cfff04a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:00.889686 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1083/2159] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:00 2022] DELETE /identity/v3/projects/502468ce2c6543c6bf1f630d874c6f22 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:00.897597 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06718b42-96f6-49b8-a14d-a2d21c4e3d43 service cinder] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:00.911718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06718b42-96f6-49b8-a14d-a2d21c4e3d43 service cinder] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:00.932179 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:00.974252 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3a59d-a1c3-493f-8100-10a1d15fc264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:00.975263 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3a59d-a1c3-493f-8100-10a1d15fc264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:00.975622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3a59d-a1c3-493f-8100-10a1d15fc264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:00.975987 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3a59d-a1c3-493f-8100-10a1d15fc264 admin admin] PATH_INFO: `/v3/users/222c4c6e0b6f4371a0d814339a50d25d` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:00.977470 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3a59d-a1c3-493f-8100-10a1d15fc264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=222c4c6e0b6f4371a0d814339a50d25d)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:00.979555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3a59d-a1c3-493f-8100-10a1d15fc264 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:01.038277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e3e3a59d-a1c3-493f-8100-10a1d15fc264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '222c4c6e0b6f4371a0d814339a50d25d'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:01.052546 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e3e3a59d-a1c3-493f-8100-10a1d15fc264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '222c4c6e0b6f4371a0d814339a50d25d'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:01.061237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e3e3a59d-a1c3-493f-8100-10a1d15fc264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '222c4c6e0b6f4371a0d814339a50d25d'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:01.062142 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e3e3a59d-a1c3-493f-8100-10a1d15fc264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '222c4c6e0b6f4371a0d814339a50d25d'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:01.062846 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e3e3a59d-a1c3-493f-8100-10a1d15fc264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '222c4c6e0b6f4371a0d814339a50d25d'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:01.078472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1077/2160] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:00 2022] DELETE /identity/v3/users/222c4c6e0b6f4371a0d814339a50d25d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:01.360437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bff43-26aa-4fbd-9278-4e855cfff04a admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:01.362176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-3c9bff43-26aa-4fbd-9278-4e855cfff04a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:01.381366 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bff43-26aa-4fbd-9278-4e855cfff04a admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:01.444314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b441cd-9e62-4e9f-8666-e3fe02be72d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:01.445213 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b441cd-9e62-4e9f-8666-e3fe02be72d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:01.445496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b441cd-9e62-4e9f-8666-e3fe02be72d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:01.445755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b441cd-9e62-4e9f-8666-e3fe02be72d4 admin admin] PATH_INFO: `/v3/projects/705d108de44b4b3f80a52b02f01dbf2d` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:01.447029 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b441cd-9e62-4e9f-8666-e3fe02be72d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=705d108de44b4b3f80a52b02f01dbf2d)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:01.448994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b441cd-9e62-4e9f-8666-e3fe02be72d4 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:01.500436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-92b441cd-9e62-4e9f-8666-e3fe02be72d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '705d108de44b4b3f80a52b02f01dbf2d', 'request_id': 'req-92b441cd-9e62-4e9f-8666-e3fe02be72d4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:01.506044 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-92b441cd-9e62-4e9f-8666-e3fe02be72d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '705d108de44b4b3f80a52b02f01dbf2d', 'request_id': 'req-92b441cd-9e62-4e9f-8666-e3fe02be72d4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:01.514812 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-92b441cd-9e62-4e9f-8666-e3fe02be72d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '705d108de44b4b3f80a52b02f01dbf2d', 'request_id': 'req-92b441cd-9e62-4e9f-8666-e3fe02be72d4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:01.527786 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-92b441cd-9e62-4e9f-8666-e3fe02be72d4 admin admin] The token cache is being invalidate because project 705d108de44b4b3f80a52b02f01dbf2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:01.528077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-92b441cd-9e62-4e9f-8666-e3fe02be72d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:01.528711 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-92b441cd-9e62-4e9f-8666-e3fe02be72d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:01.530300 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1084/2161] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:01 2022] DELETE /identity/v3/projects/705d108de44b4b3f80a52b02f01dbf2d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:01.534457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3a59d-a1c3-493f-8100-10a1d15fc264 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:01.536287 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-e3e3a59d-a1c3-493f-8100-10a1d15fc264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:01.549903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3a59d-a1c3-493f-8100-10a1d15fc264 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:01.608649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501757bb-3816-4e3a-ba3e-47f646f3a9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:01.609452 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501757bb-3816-4e3a-ba3e-47f646f3a9f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:01.609734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501757bb-3816-4e3a-ba3e-47f646f3a9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:01.609996 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501757bb-3816-4e3a-ba3e-47f646f3a9f3 admin admin] PATH_INFO: `/v3/users/e58588a7dc4d4c53a12b6bf454aa515f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:01.611308 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501757bb-3816-4e3a-ba3e-47f646f3a9f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e58588a7dc4d4c53a12b6bf454aa515f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:01.613312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501757bb-3816-4e3a-ba3e-47f646f3a9f3 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:01.662624 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:01.665337 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-501757bb-3816-4e3a-ba3e-47f646f3a9f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e58588a7dc4d4c53a12b6bf454aa515f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:01.678597 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-501757bb-3816-4e3a-ba3e-47f646f3a9f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e58588a7dc4d4c53a12b6bf454aa515f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:01.685404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-501757bb-3816-4e3a-ba3e-47f646f3a9f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e58588a7dc4d4c53a12b6bf454aa515f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:01.686055 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-501757bb-3816-4e3a-ba3e-47f646f3a9f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e58588a7dc4d4c53a12b6bf454aa515f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:01.686706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-501757bb-3816-4e3a-ba3e-47f646f3a9f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e58588a7dc4d4c53a12b6bf454aa515f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:01.702054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1078/2162] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:01 2022] DELETE /identity/v3/users/e58588a7dc4d4c53a12b6bf454aa515f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:01.991713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b441cd-9e62-4e9f-8666-e3fe02be72d4 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:01.993431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-92b441cd-9e62-4e9f-8666-e3fe02be72d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:02.008893 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b441cd-9e62-4e9f-8666-e3fe02be72d4 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:02.089202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f74ab-b830-47b9-b0b4-6cebae05a093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:02.090086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f74ab-b830-47b9-b0b4-6cebae05a093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:02.090370 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f74ab-b830-47b9-b0b4-6cebae05a093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:02.090644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f74ab-b830-47b9-b0b4-6cebae05a093 admin admin] PATH_INFO: `/v3/projects/061fd45b758d44fabde4571a13571210` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:02.091897 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f74ab-b830-47b9-b0b4-6cebae05a093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=061fd45b758d44fabde4571a13571210)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:02.094005 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f74ab-b830-47b9-b0b4-6cebae05a093 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:02.130455 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:02.150099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e23f74ab-b830-47b9-b0b4-6cebae05a093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '061fd45b758d44fabde4571a13571210', 'request_id': 'req-e23f74ab-b830-47b9-b0b4-6cebae05a093'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:02.155571 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e23f74ab-b830-47b9-b0b4-6cebae05a093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '061fd45b758d44fabde4571a13571210', 'request_id': 'req-e23f74ab-b830-47b9-b0b4-6cebae05a093'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:02.165518 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e23f74ab-b830-47b9-b0b4-6cebae05a093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '061fd45b758d44fabde4571a13571210', 'request_id': 'req-e23f74ab-b830-47b9-b0b4-6cebae05a093'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:02.177323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e23f74ab-b830-47b9-b0b4-6cebae05a093 admin admin] The token cache is being invalidate because project 061fd45b758d44fabde4571a13571210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:02.177608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e23f74ab-b830-47b9-b0b4-6cebae05a093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:02.178253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e23f74ab-b830-47b9-b0b4-6cebae05a093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:02.179964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1085/2163] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:01 2022] DELETE /identity/v3/projects/061fd45b758d44fabde4571a13571210 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:45:02.186055 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6525a-f34f-4e93-abc7-ff5cb5201b67 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:02.186557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6525a-f34f-4e93-abc7-ff5cb5201b67 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:02.186742 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6525a-f34f-4e93-abc7-ff5cb5201b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:02.206531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-b4b6525a-f34f-4e93-abc7-ff5cb5201b67 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:02.207811 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b4b6525a-f34f-4e93-abc7-ff5cb5201b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:02.272822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1079/2164] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:02.281559 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be052172-f903-4675-98a3-1e645842b8e8 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:02.281824 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be052172-f903-4675-98a3-1e645842b8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:02.282126 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be052172-f903-4675-98a3-1e645842b8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:02.302186 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-be052172-f903-4675-98a3-1e645842b8e8 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:02.303518 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-be052172-f903-4675-98a3-1e645842b8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:02.313329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1086/2165] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:02.317919 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6525a-f34f-4e93-abc7-ff5cb5201b67 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:02.328703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6525a-f34f-4e93-abc7-ff5cb5201b67 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:02.334040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2224b-1d52-42a1-9856-211f0456a9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:02.334739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2224b-1d52-42a1-9856-211f0456a9ac admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:02.335039 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2224b-1d52-42a1-9856-211f0456a9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:02.335335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2224b-1d52-42a1-9856-211f0456a9ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:02.336075 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de2224b-1d52-42a1-9856-211f0456a9ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:45:02.336385 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de2224b-1d52-42a1-9856-211f0456a9ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:02.338027 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de2224b-1d52-42a1-9856-211f0456a9ac admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:02.352032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1080/2166] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:45:02.356350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be052172-f903-4675-98a3-1e645842b8e8 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:02.366399 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be052172-f903-4675-98a3-1e645842b8e8 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:02.371723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91441fea-db29-40ac-9e73-36cf077257ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:02.372332 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91441fea-db29-40ac-9e73-36cf077257ef admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:02.372615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91441fea-db29-40ac-9e73-36cf077257ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:02.372885 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91441fea-db29-40ac-9e73-36cf077257ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:02.373581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91441fea-db29-40ac-9e73-36cf077257ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:02.375539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91441fea-db29-40ac-9e73-36cf077257ef admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:02.406966 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1087/2167] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:45:02 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:02.408192 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2224b-1d52-42a1-9856-211f0456a9ac admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:02.419525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2224b-1d52-42a1-9856-211f0456a9ac admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:02.478989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d9a76-9ae1-4443-b05c-e9e0d9cf4de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:02.479602 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d9a76-9ae1-4443-b05c-e9e0d9cf4de4 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:02.479894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d9a76-9ae1-4443-b05c-e9e0d9cf4de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:02.480197 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d9a76-9ae1-4443-b05c-e9e0d9cf4de4 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:02.480901 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d9a76-9ae1-4443-b05c-e9e0d9cf4de4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:02.483288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d9a76-9ae1-4443-b05c-e9e0d9cf4de4 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:02.508279 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1081/2168] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:45:02 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:02.509659 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91441fea-db29-40ac-9e73-36cf077257ef admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:02.525005 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91441fea-db29-40ac-9e73-36cf077257ef admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:02.530461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce363494-20ff-4454-8f57-408458225bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:02.531170 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce363494-20ff-4454-8f57-408458225bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:02.531458 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce363494-20ff-4454-8f57-408458225bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:02.531736 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce363494-20ff-4454-8f57-408458225bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:02.532654 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce363494-20ff-4454-8f57-408458225bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:02.534423 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce363494-20ff-4454-8f57-408458225bf1 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:02.547791 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1088/2169] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:02.552114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d9a76-9ae1-4443-b05c-e9e0d9cf4de4 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:02.562648 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d9a76-9ae1-4443-b05c-e9e0d9cf4de4 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:02.567451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5bee8-0fda-4c32-a477-0fc1bf7d4ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:02.568124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5bee8-0fda-4c32-a477-0fc1bf7d4ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:02.568409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5bee8-0fda-4c32-a477-0fc1bf7d4ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:02.568685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5bee8-0fda-4c32-a477-0fc1bf7d4ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:02.569426 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5bee8-0fda-4c32-a477-0fc1bf7d4ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:02.571089 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5bee8-0fda-4c32-a477-0fc1bf7d4ee0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:02.582374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1082/2170] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:02.586706 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce363494-20ff-4454-8f57-408458225bf1 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:02.597405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce363494-20ff-4454-8f57-408458225bf1 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:02.602660 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98158668-96d4-4ec4-9a85-02f4b2898dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:02.603655 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98158668-96d4-4ec4-9a85-02f4b2898dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:02.603979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98158668-96d4-4ec4-9a85-02f4b2898dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:02.604335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98158668-96d4-4ec4-9a85-02f4b2898dc9 admin admin] PATH_INFO: `/v3/projects/7845b6b63db14fc9be25dbc720b08eda/users/fac690a92b314d878e873e82cd0bb894/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:02.621209 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98158668-96d4-4ec4-9a85-02f4b2898dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7845b6b63db14fc9be25dbc720b08eda, user_id=fac690a92b314d878e873e82cd0bb894, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:02.624434 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98158668-96d4-4ec4-9a85-02f4b2898dc9 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:02.639267 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1089/2171] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:02 2022] PUT /identity/v3/projects/7845b6b63db14fc9be25dbc720b08eda/users/fac690a92b314d878e873e82cd0bb894/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:02.653283 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f5614-2a10-4956-b6d6-b4d1124d4928 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:02.653751 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f5614-2a10-4956-b6d6-b4d1124d4928 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:02.654407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f5614-2a10-4956-b6d6-b4d1124d4928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:02.683932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-310f5614-2a10-4956-b6d6-b4d1124d4928 None None] MFA Rules not processed for user `fac690a92b314d878e873e82cd0bb894`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:02.685151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-310f5614-2a10-4956-b6d6-b4d1124d4928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:02.699328 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:02.752998 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1083/2172] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:02.757909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98158668-96d4-4ec4-9a85-02f4b2898dc9 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:02.821335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98158668-96d4-4ec4-9a85-02f4b2898dc9 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:02.826439 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70cd43-a6c1-410b-9f4b-46f690e0bc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:02.827202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70cd43-a6c1-410b-9f4b-46f690e0bc3d admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:02.827490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70cd43-a6c1-410b-9f4b-46f690e0bc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:02.827764 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b70cd43-a6c1-410b-9f4b-46f690e0bc3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:02.828537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70cd43-a6c1-410b-9f4b-46f690e0bc3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:02.830560 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b70cd43-a6c1-410b-9f4b-46f690e0bc3d admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:02.856770 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1090/2173] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:45:02 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:45:02.861077 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f5614-2a10-4956-b6d6-b4d1124d4928 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:02.872835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f5614-2a10-4956-b6d6-b4d1124d4928 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:02.929378 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12807ee-96b0-40db-8b31-1966daf9484a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:02.930060 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12807ee-96b0-40db-8b31-1966daf9484a admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:02.930345 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12807ee-96b0-40db-8b31-1966daf9484a admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:02.930622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12807ee-96b0-40db-8b31-1966daf9484a admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:02.931373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12807ee-96b0-40db-8b31-1966daf9484a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:02.933375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12807ee-96b0-40db-8b31-1966daf9484a admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:02.955311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1084/2174] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:45:02 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:02.959934 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70cd43-a6c1-410b-9f4b-46f690e0bc3d admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:02.972448 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b70cd43-a6c1-410b-9f4b-46f690e0bc3d admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:02.977545 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5e03a-1b62-40f5-be44-739381e6f36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:02.978218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5e03a-1b62-40f5-be44-739381e6f36f admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:02.978528 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5e03a-1b62-40f5-be44-739381e6f36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:02.978810 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5e03a-1b62-40f5-be44-739381e6f36f admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:02.979589 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5e03a-1b62-40f5-be44-739381e6f36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:02.981234 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5e03a-1b62-40f5-be44-739381e6f36f admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:02.992700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1091/2175] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:02.996798 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12807ee-96b0-40db-8b31-1966daf9484a admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:03.011333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12807ee-96b0-40db-8b31-1966daf9484a admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:03.019582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4516abe-4cdd-48e2-9570-7b37b9d3dc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:03.020241 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4516abe-4cdd-48e2-9570-7b37b9d3dc58 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:03.020553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4516abe-4cdd-48e2-9570-7b37b9d3dc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:03.020828 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4516abe-4cdd-48e2-9570-7b37b9d3dc58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:03.021591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4516abe-4cdd-48e2-9570-7b37b9d3dc58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:03.023253 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4516abe-4cdd-48e2-9570-7b37b9d3dc58 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:03.034097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1085/2176] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:03.038542 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5e03a-1b62-40f5-be44-739381e6f36f admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:03.049427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5e03a-1b62-40f5-be44-739381e6f36f admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:03.054722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424940b1-0d3a-4159-b702-eea306f5e259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:03.055611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424940b1-0d3a-4159-b702-eea306f5e259 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:03.055894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424940b1-0d3a-4159-b702-eea306f5e259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:03.056167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424940b1-0d3a-4159-b702-eea306f5e259 admin admin] PATH_INFO: `/v3/projects/b123cefb46304e5a9f09eee3770a1978/users/ba0744a7da2c41c194bbbe8774bbaeb7/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:03.077763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424940b1-0d3a-4159-b702-eea306f5e259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b123cefb46304e5a9f09eee3770a1978, user_id=ba0744a7da2c41c194bbbe8774bbaeb7, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:03.084324 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424940b1-0d3a-4159-b702-eea306f5e259 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:03.100633 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1092/2177] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:03 2022] PUT /identity/v3/projects/b123cefb46304e5a9f09eee3770a1978/users/ba0744a7da2c41c194bbbe8774bbaeb7/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:03.114862 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435442b2-5bfb-426a-bddb-ffb7c597de06 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:03.115240 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435442b2-5bfb-426a-bddb-ffb7c597de06 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:03.115443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435442b2-5bfb-426a-bddb-ffb7c597de06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:03.143320 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-435442b2-5bfb-426a-bddb-ffb7c597de06 None None] MFA Rules not processed for user `ba0744a7da2c41c194bbbe8774bbaeb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:03.144543 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-435442b2-5bfb-426a-bddb-ffb7c597de06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:03.221069 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1086/2178] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:03.225727 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424940b1-0d3a-4159-b702-eea306f5e259 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:03.237399 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:03.299888 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424940b1-0d3a-4159-b702-eea306f5e259 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:03.305260 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201f10c-d48e-43ef-ade3-3a092578b4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:03.305960 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201f10c-d48e-43ef-ade3-3a092578b4be admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:03.306274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201f10c-d48e-43ef-ade3-3a092578b4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:03.306681 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201f10c-d48e-43ef-ade3-3a092578b4be admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:03.307433 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201f10c-d48e-43ef-ade3-3a092578b4be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:03.309501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201f10c-d48e-43ef-ade3-3a092578b4be admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:03.339607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1093/2179] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:45:03 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:03.343857 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435442b2-5bfb-426a-bddb-ffb7c597de06 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:03.355455 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435442b2-5bfb-426a-bddb-ffb7c597de06 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:03.391105 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:03.434946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da235c-2f7b-4e7b-ac03-636a573061b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:03.435784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da235c-2f7b-4e7b-ac03-636a573061b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:03.436072 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da235c-2f7b-4e7b-ac03-636a573061b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:03.436351 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da235c-2f7b-4e7b-ac03-636a573061b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:03.437096 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da235c-2f7b-4e7b-ac03-636a573061b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:03.439184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da235c-2f7b-4e7b-ac03-636a573061b2 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:03.464972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1087/2180] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:45:03 2022] POST /identity/v3/users => generated 299 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:45:03.469064 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201f10c-d48e-43ef-ade3-3a092578b4be admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:03.480119 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201f10c-d48e-43ef-ade3-3a092578b4be admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:03.484776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8521a-4cb8-48ed-81f6-76b16c01f087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:03.485484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8521a-4cb8-48ed-81f6-76b16c01f087 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:03.485779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8521a-4cb8-48ed-81f6-76b16c01f087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:03.486052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8521a-4cb8-48ed-81f6-76b16c01f087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:03.486901 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8521a-4cb8-48ed-81f6-76b16c01f087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:03.488551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8521a-4cb8-48ed-81f6-76b16c01f087 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:03.499598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1094/2181] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:03.503889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da235c-2f7b-4e7b-ac03-636a573061b2 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:03.514290 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da235c-2f7b-4e7b-ac03-636a573061b2 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:03.520070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976151a-30e1-422b-a715-5451fbe0d7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:03.520976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976151a-30e1-422b-a715-5451fbe0d7ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:03.521257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976151a-30e1-422b-a715-5451fbe0d7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:03.521550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976151a-30e1-422b-a715-5451fbe0d7ae admin admin] PATH_INFO: `/v3/projects/c8633ff85f9c4510a949645436a0210e/users/416d44c244a44e4bae52dc3c1bc7e285/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:03.551685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7976151a-30e1-422b-a715-5451fbe0d7ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8633ff85f9c4510a949645436a0210e, user_id=416d44c244a44e4bae52dc3c1bc7e285, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:03.554515 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7976151a-30e1-422b-a715-5451fbe0d7ae admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:03.576882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1088/2182] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:03 2022] PUT /identity/v3/projects/c8633ff85f9c4510a949645436a0210e/users/416d44c244a44e4bae52dc3c1bc7e285/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:45:03.581025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8521a-4cb8-48ed-81f6-76b16c01f087 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:03.591525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8521a-4cb8-48ed-81f6-76b16c01f087 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:03.657446 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9337984-5336-43a5-9586-0e71970a85a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:03.658251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9337984-5336-43a5-9586-0e71970a85a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:03.658638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9337984-5336-43a5-9586-0e71970a85a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:03.659051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9337984-5336-43a5-9586-0e71970a85a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:03.660262 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9337984-5336-43a5-9586-0e71970a85a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:03.662024 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9337984-5336-43a5-9586-0e71970a85a6 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:03.676249 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1095/2183] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:03 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:03.677709 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976151a-30e1-422b-a715-5451fbe0d7ae admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:03.688403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976151a-30e1-422b-a715-5451fbe0d7ae admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:03.693020 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69196b08-5ffc-4604-b21a-6accf73dfb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:03.693919 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69196b08-5ffc-4604-b21a-6accf73dfb88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:03.694239 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69196b08-5ffc-4604-b21a-6accf73dfb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:03.694538 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69196b08-5ffc-4604-b21a-6accf73dfb88 admin admin] PATH_INFO: `/v3/projects/c8633ff85f9c4510a949645436a0210e/users/416d44c244a44e4bae52dc3c1bc7e285/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:03.696740 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69196b08-5ffc-4604-b21a-6accf73dfb88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8633ff85f9c4510a949645436a0210e, user_id=416d44c244a44e4bae52dc3c1bc7e285, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:03.699631 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69196b08-5ffc-4604-b21a-6accf73dfb88 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:03.710897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1089/2184] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:03 2022] PUT /identity/v3/projects/c8633ff85f9c4510a949645436a0210e/users/416d44c244a44e4bae52dc3c1bc7e285/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:03.721212 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747cd70-bdf9-4225-b22b-3bfd29fbe3d7 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:03.721510 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747cd70-bdf9-4225-b22b-3bfd29fbe3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:03.721795 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747cd70-bdf9-4225-b22b-3bfd29fbe3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:03.751073 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-d747cd70-bdf9-4225-b22b-3bfd29fbe3d7 None None] MFA Rules not processed for user `416d44c244a44e4bae52dc3c1bc7e285`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:03.752370 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d747cd70-bdf9-4225-b22b-3bfd29fbe3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:03.836883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:03.842247 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1096/2185] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:03.858348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69196b08-5ffc-4604-b21a-6accf73dfb88 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:03.860380 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-69196b08-5ffc-4604-b21a-6accf73dfb88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:03.921376 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69196b08-5ffc-4604-b21a-6accf73dfb88 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:03.925065 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bc8e8-7b04-4eb9-be64-14799b87678b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:03.925718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bc8e8-7b04-4eb9-be64-14799b87678b service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:03.926002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bc8e8-7b04-4eb9-be64-14799b87678b service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:03.926288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bc8e8-7b04-4eb9-be64-14799b87678b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:03.986467 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bc8e8-7b04-4eb9-be64-14799b87678b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:03.988661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bc8e8-7b04-4eb9-be64-14799b87678b service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:04.002676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1090/2186] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:07.651285 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747cd70-bdf9-4225-b22b-3bfd29fbe3d7 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:07.652956 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d747cd70-bdf9-4225-b22b-3bfd29fbe3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:07.666584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747cd70-bdf9-4225-b22b-3bfd29fbe3d7 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:07.670382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c450a2c-f06e-4e29-b7a3-da42a5229167 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:07.671077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c450a2c-f06e-4e29-b7a3-da42a5229167 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:07.671364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c450a2c-f06e-4e29-b7a3-da42a5229167 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:07.671638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c450a2c-f06e-4e29-b7a3-da42a5229167 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:07.729804 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c450a2c-f06e-4e29-b7a3-da42a5229167 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:07.731310 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c450a2c-f06e-4e29-b7a3-da42a5229167 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:07.740877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1097/2187] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:08.310053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bc8e8-7b04-4eb9-be64-14799b87678b service cinder] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:08.321250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bc8e8-7b04-4eb9-be64-14799b87678b service cinder] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:08.324968 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c31d21-0b8d-4119-a1fb-08383f21f811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:08.325626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c31d21-0b8d-4119-a1fb-08383f21f811 service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:08.325909 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c31d21-0b8d-4119-a1fb-08383f21f811 service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:08.326211 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c31d21-0b8d-4119-a1fb-08383f21f811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:08.335928 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c31d21-0b8d-4119-a1fb-08383f21f811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:08.337439 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c31d21-0b8d-4119-a1fb-08383f21f811 service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:08.347359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1091/2188] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:15.488237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928aa3c-9149-42e7-9efc-629db61b2a53 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:15.488729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928aa3c-9149-42e7-9efc-629db61b2a53 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:15.488839 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928aa3c-9149-42e7-9efc-629db61b2a53 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:15.490433 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1098/2189] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jul 21 05:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:45:15.509338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16ea66-dc15-4c10-8fdd-70e86b508a94 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:15.509872 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16ea66-dc15-4c10-8fdd-70e86b508a94 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:15.509956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16ea66-dc15-4c10-8fdd-70e86b508a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:15.536939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-3c16ea66-dc15-4c10-8fdd-70e86b508a94 None None] MFA Rules not processed for user `03d868598e39448881b43d299253d821`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:15.538212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3c16ea66-dc15-4c10-8fdd-70e86b508a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:15.556958 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:15.616290 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1092/2190] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jul 21 05:45:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:15.628454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928aa3c-9149-42e7-9efc-629db61b2a53 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:15.631120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a928aa3c-9149-42e7-9efc-629db61b2a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:15.648509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928aa3c-9149-42e7-9efc-629db61b2a53 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:15.653832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0349b-97bd-40e7-952a-2a5a5c77026c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:15.654549 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0349b-97bd-40e7-952a-2a5a5c77026c service placement] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:15.654908 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0349b-97bd-40e7-952a-2a5a5c77026c service placement] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:15.655292 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0349b-97bd-40e7-952a-2a5a5c77026c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:15.672091 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0349b-97bd-40e7-952a-2a5a5c77026c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:15.673731 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0349b-97bd-40e7-952a-2a5a5c77026c service placement] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:15.683769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1099/2191] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:16.354940 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16ea66-dc15-4c10-8fdd-70e86b508a94 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:16.356547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3c16ea66-dc15-4c10-8fdd-70e86b508a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:16.434294 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16ea66-dc15-4c10-8fdd-70e86b508a94 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:16.439943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f880494-d99e-4e07-92ca-0a9689f9c844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:16.440697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f880494-d99e-4e07-92ca-0a9689f9c844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:16.442514 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f880494-d99e-4e07-92ca-0a9689f9c844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:16.442817 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f880494-d99e-4e07-92ca-0a9689f9c844 admin admin] PATH_INFO: `/v3/users/6278f41b27c24139b647b23b1f2df406` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:16.444104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f880494-d99e-4e07-92ca-0a9689f9c844 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6278f41b27c24139b647b23b1f2df406)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:16.446102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f880494-d99e-4e07-92ca-0a9689f9c844 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:16.500236 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-4f880494-d99e-4e07-92ca-0a9689f9c844 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6278f41b27c24139b647b23b1f2df406'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:16.514257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-4f880494-d99e-4e07-92ca-0a9689f9c844 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6278f41b27c24139b647b23b1f2df406'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:16.522353 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-4f880494-d99e-4e07-92ca-0a9689f9c844 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6278f41b27c24139b647b23b1f2df406'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:16.523137 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-4f880494-d99e-4e07-92ca-0a9689f9c844 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6278f41b27c24139b647b23b1f2df406'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:16.523748 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-4f880494-d99e-4e07-92ca-0a9689f9c844 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6278f41b27c24139b647b23b1f2df406'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:16.549817 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1093/2192] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:16 2022] DELETE /identity/v3/users/6278f41b27c24139b647b23b1f2df406 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:16.732409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0349b-97bd-40e7-952a-2a5a5c77026c service placement] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:16.743687 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0349b-97bd-40e7-952a-2a5a5c77026c service placement] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:16.803689 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47acdcec-8303-45c2-8d07-4648611658fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:16.805019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acdcec-8303-45c2-8d07-4648611658fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:16.805386 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acdcec-8303-45c2-8d07-4648611658fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:16.805666 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acdcec-8303-45c2-8d07-4648611658fc admin admin] PATH_INFO: `/v3/projects/2052497ccd9a4343a388cc2d0b90425e` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:16.806865 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47acdcec-8303-45c2-8d07-4648611658fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2052497ccd9a4343a388cc2d0b90425e)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:16.808888 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47acdcec-8303-45c2-8d07-4648611658fc admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:16.861665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-47acdcec-8303-45c2-8d07-4648611658fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2052497ccd9a4343a388cc2d0b90425e', 'request_id': 'req-47acdcec-8303-45c2-8d07-4648611658fc'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:16.867778 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-47acdcec-8303-45c2-8d07-4648611658fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2052497ccd9a4343a388cc2d0b90425e', 'request_id': 'req-47acdcec-8303-45c2-8d07-4648611658fc'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:16.877741 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-47acdcec-8303-45c2-8d07-4648611658fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2052497ccd9a4343a388cc2d0b90425e', 'request_id': 'req-47acdcec-8303-45c2-8d07-4648611658fc'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:16.891022 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-47acdcec-8303-45c2-8d07-4648611658fc admin admin] The token cache is being invalidate because project 2052497ccd9a4343a388cc2d0b90425e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:16.891412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-47acdcec-8303-45c2-8d07-4648611658fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:16.892139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-47acdcec-8303-45c2-8d07-4648611658fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:16.894340 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1100/2193] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:16 2022] DELETE /identity/v3/projects/2052497ccd9a4343a388cc2d0b90425e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:16.898578 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f880494-d99e-4e07-92ca-0a9689f9c844 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:16.900278 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4f880494-d99e-4e07-92ca-0a9689f9c844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:16.917009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f880494-d99e-4e07-92ca-0a9689f9c844 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:16.954777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:17.006950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590afd7-e184-4746-83d0-e84be7b4c889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:17.008278 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590afd7-e184-4746-83d0-e84be7b4c889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:17.008921 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590afd7-e184-4746-83d0-e84be7b4c889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:17.009440 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590afd7-e184-4746-83d0-e84be7b4c889 admin admin] PATH_INFO: `/v3/users/66a2d6b32e5945a188de043ea7fa73cf` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:17.021726 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590afd7-e184-4746-83d0-e84be7b4c889 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66a2d6b32e5945a188de043ea7fa73cf)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:17.024354 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590afd7-e184-4746-83d0-e84be7b4c889 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:17.093091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b590afd7-e184-4746-83d0-e84be7b4c889 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66a2d6b32e5945a188de043ea7fa73cf'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:17.105523 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b590afd7-e184-4746-83d0-e84be7b4c889 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66a2d6b32e5945a188de043ea7fa73cf'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:17.114432 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b590afd7-e184-4746-83d0-e84be7b4c889 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66a2d6b32e5945a188de043ea7fa73cf'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:17.115797 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b590afd7-e184-4746-83d0-e84be7b4c889 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66a2d6b32e5945a188de043ea7fa73cf'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:17.116355 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b590afd7-e184-4746-83d0-e84be7b4c889 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66a2d6b32e5945a188de043ea7fa73cf'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:17.145852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1094/2194] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:16 2022] DELETE /identity/v3/users/66a2d6b32e5945a188de043ea7fa73cf => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:17.535657 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47acdcec-8303-45c2-8d07-4648611658fc admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:17.537307 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-47acdcec-8303-45c2-8d07-4648611658fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:17.555700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47acdcec-8303-45c2-8d07-4648611658fc admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:17.585075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:17.699303 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b6eae-40f4-4792-8f4e-18e75b76fd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:17.700198 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b6eae-40f4-4792-8f4e-18e75b76fd5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:17.700490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b6eae-40f4-4792-8f4e-18e75b76fd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:17.700770 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b6eae-40f4-4792-8f4e-18e75b76fd5e admin admin] PATH_INFO: `/v3/projects/4388527b12654334993397036bbb6f69` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:17.701989 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b6eae-40f4-4792-8f4e-18e75b76fd5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4388527b12654334993397036bbb6f69)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:17.704034 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b6eae-40f4-4792-8f4e-18e75b76fd5e admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:17.755855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-660b6eae-40f4-4792-8f4e-18e75b76fd5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4388527b12654334993397036bbb6f69', 'request_id': 'req-660b6eae-40f4-4792-8f4e-18e75b76fd5e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:17.761814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-660b6eae-40f4-4792-8f4e-18e75b76fd5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4388527b12654334993397036bbb6f69', 'request_id': 'req-660b6eae-40f4-4792-8f4e-18e75b76fd5e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:17.770737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-660b6eae-40f4-4792-8f4e-18e75b76fd5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4388527b12654334993397036bbb6f69', 'request_id': 'req-660b6eae-40f4-4792-8f4e-18e75b76fd5e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:17.783611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-660b6eae-40f4-4792-8f4e-18e75b76fd5e admin admin] The token cache is being invalidate because project 4388527b12654334993397036bbb6f69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:17.783948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-660b6eae-40f4-4792-8f4e-18e75b76fd5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:17.785051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-660b6eae-40f4-4792-8f4e-18e75b76fd5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:17.786924 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1101/2195] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:17 2022] DELETE /identity/v3/projects/4388527b12654334993397036bbb6f69 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:17.792256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893db884-19f3-46d5-a1e4-076718730f74 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:17.792705 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893db884-19f3-46d5-a1e4-076718730f74 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:17.792821 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893db884-19f3-46d5-a1e4-076718730f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:17.814706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-893db884-19f3-46d5-a1e4-076718730f74 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:17.816019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-893db884-19f3-46d5-a1e4-076718730f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:17.880278 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1095/2196] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:17.889405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719723c6-5c45-4fa9-a832-fcd622fb9935 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:17.889625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719723c6-5c45-4fa9-a832-fcd622fb9935 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:17.889910 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719723c6-5c45-4fa9-a832-fcd622fb9935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:17.909294 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-719723c6-5c45-4fa9-a832-fcd622fb9935 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:17.910885 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-719723c6-5c45-4fa9-a832-fcd622fb9935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:17.919147 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1102/2197] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:17.923675 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893db884-19f3-46d5-a1e4-076718730f74 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:17.934193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893db884-19f3-46d5-a1e4-076718730f74 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:17.939081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f3f5b-5de1-4652-bd8f-f7051eff8d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:17.939699 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f3f5b-5de1-4652-bd8f-f7051eff8d3d admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:17.940016 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f3f5b-5de1-4652-bd8f-f7051eff8d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:17.940294 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f3f5b-5de1-4652-bd8f-f7051eff8d3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:17.941028 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f3f5b-5de1-4652-bd8f-f7051eff8d3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:45:17.941338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f3f5b-5de1-4652-bd8f-f7051eff8d3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:17.942944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f3f5b-5de1-4652-bd8f-f7051eff8d3d admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:17.954991 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1096/2198] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:45:17.959499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719723c6-5c45-4fa9-a832-fcd622fb9935 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:17.969033 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719723c6-5c45-4fa9-a832-fcd622fb9935 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:17.973965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cbd7e5-5224-4bc5-b722-52698d249dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:17.974713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbd7e5-5224-4bc5-b722-52698d249dbb admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:17.975122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbd7e5-5224-4bc5-b722-52698d249dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:17.975875 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbd7e5-5224-4bc5-b722-52698d249dbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:17.976603 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cbd7e5-5224-4bc5-b722-52698d249dbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:17.978875 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cbd7e5-5224-4bc5-b722-52698d249dbb admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:18.017834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1103/2199] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:45:17 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:45:18.023271 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f3f5b-5de1-4652-bd8f-f7051eff8d3d admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:18.034947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f3f5b-5de1-4652-bd8f-f7051eff8d3d admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:18.093136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:18.109488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df87ba-57c7-44cf-98d8-7614c09f7781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:18.110228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df87ba-57c7-44cf-98d8-7614c09f7781 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:18.110518 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df87ba-57c7-44cf-98d8-7614c09f7781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:18.110794 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df87ba-57c7-44cf-98d8-7614c09f7781 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:18.111548 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df87ba-57c7-44cf-98d8-7614c09f7781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:18.113544 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df87ba-57c7-44cf-98d8-7614c09f7781 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:18.138285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1097/2200] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:45:18 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:18.142440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cbd7e5-5224-4bc5-b722-52698d249dbb admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:18.153215 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cbd7e5-5224-4bc5-b722-52698d249dbb admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:18.157767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167121d9-112e-4bef-bc94-9bcd7bc782fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:18.158451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167121d9-112e-4bef-bc94-9bcd7bc782fb admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:18.158858 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167121d9-112e-4bef-bc94-9bcd7bc782fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:18.159181 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167121d9-112e-4bef-bc94-9bcd7bc782fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:18.160271 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167121d9-112e-4bef-bc94-9bcd7bc782fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:18.162045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167121d9-112e-4bef-bc94-9bcd7bc782fb admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:18.173722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1104/2201] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:45:18.177677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df87ba-57c7-44cf-98d8-7614c09f7781 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:18.188327 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df87ba-57c7-44cf-98d8-7614c09f7781 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:18.193421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce58ec-b245-424c-bc0b-e4540ab6334f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:18.194098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce58ec-b245-424c-bc0b-e4540ab6334f admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:18.194377 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce58ec-b245-424c-bc0b-e4540ab6334f admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:18.194646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce58ec-b245-424c-bc0b-e4540ab6334f admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:18.195410 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce58ec-b245-424c-bc0b-e4540ab6334f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:18.197015 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce58ec-b245-424c-bc0b-e4540ab6334f admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:18.211007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1098/2202] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:45:18.212069 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167121d9-112e-4bef-bc94-9bcd7bc782fb admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:18.222507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167121d9-112e-4bef-bc94-9bcd7bc782fb admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:18.227330 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18dff5-6ab8-4902-a417-d36623e6be74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:18.228211 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18dff5-6ab8-4902-a417-d36623e6be74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:18.228490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18dff5-6ab8-4902-a417-d36623e6be74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:18.228791 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18dff5-6ab8-4902-a417-d36623e6be74 admin admin] PATH_INFO: `/v3/projects/7647815f455e44eca4a70921341c37df/users/1220951d701745ecb80e75f9eb92863a/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:18.246037 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18dff5-6ab8-4902-a417-d36623e6be74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7647815f455e44eca4a70921341c37df, user_id=1220951d701745ecb80e75f9eb92863a, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:18.249249 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18dff5-6ab8-4902-a417-d36623e6be74 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:18.271779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1105/2203] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:18 2022] PUT /identity/v3/projects/7647815f455e44eca4a70921341c37df/users/1220951d701745ecb80e75f9eb92863a/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:45:18.282893 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dcacc-9ea4-4fbd-b3d4-aa3b6fc2043c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:18.283445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dcacc-9ea4-4fbd-b3d4-aa3b6fc2043c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:18.283820 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2dcacc-9ea4-4fbd-b3d4-aa3b6fc2043c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:18.320419 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-8d2dcacc-9ea4-4fbd-b3d4-aa3b6fc2043c None None] MFA Rules not processed for user `1220951d701745ecb80e75f9eb92863a`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:18.321667 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8d2dcacc-9ea4-4fbd-b3d4-aa3b6fc2043c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:18.405024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1099/2204] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:45:18.413457 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18dff5-6ab8-4902-a417-d36623e6be74 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:18.415299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-9b18dff5-6ab8-4902-a417-d36623e6be74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:18.469445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:18.482158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18dff5-6ab8-4902-a417-d36623e6be74 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:18.488891 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51046a-8451-4481-b54f-d8d45f90d2ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:18.489660 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51046a-8451-4481-b54f-d8d45f90d2ee service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:18.490072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51046a-8451-4481-b54f-d8d45f90d2ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:18.490560 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51046a-8451-4481-b54f-d8d45f90d2ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:18.500851 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51046a-8451-4481-b54f-d8d45f90d2ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:18.502403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51046a-8451-4481-b54f-d8d45f90d2ee service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:18.526082 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1106/2205] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:18.539031 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2dcacc-9ea4-4fbd-b3d4-aa3b6fc2043c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:18.614966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2dcacc-9ea4-4fbd-b3d4-aa3b6fc2043c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:18.621341 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf0f16-907a-431d-9e06-81cdf0155d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:18.622356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf0f16-907a-431d-9e06-81cdf0155d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:18.622729 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf0f16-907a-431d-9e06-81cdf0155d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:18.623136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf0f16-907a-431d-9e06-81cdf0155d8b admin admin] PATH_INFO: `/v3/users/1220951d701745ecb80e75f9eb92863a` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:18.624467 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf0f16-907a-431d-9e06-81cdf0155d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1220951d701745ecb80e75f9eb92863a)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:18.626994 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf0f16-907a-431d-9e06-81cdf0155d8b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:18.726838 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-08bf0f16-907a-431d-9e06-81cdf0155d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1220951d701745ecb80e75f9eb92863a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:18.738525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:18.756410 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-08bf0f16-907a-431d-9e06-81cdf0155d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1220951d701745ecb80e75f9eb92863a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:18.775611 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-08bf0f16-907a-431d-9e06-81cdf0155d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1220951d701745ecb80e75f9eb92863a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:18.776483 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-08bf0f16-907a-431d-9e06-81cdf0155d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1220951d701745ecb80e75f9eb92863a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:18.777212 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-08bf0f16-907a-431d-9e06-81cdf0155d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1220951d701745ecb80e75f9eb92863a'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:18.803875 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1100/2206] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:18 2022] DELETE /identity/v3/users/1220951d701745ecb80e75f9eb92863a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:18.808947 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aed2ae-07dc-4e07-9203-95ca69c90f96 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:18.809730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aed2ae-07dc-4e07-9203-95ca69c90f96 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:18.810320 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aed2ae-07dc-4e07-9203-95ca69c90f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:18.841093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-10aed2ae-07dc-4e07-9203-95ca69c90f96 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:18.843141 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-10aed2ae-07dc-4e07-9203-95ca69c90f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:18.958899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1107/2207] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:18.960216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf0f16-907a-431d-9e06-81cdf0155d8b admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:18.962455 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-08bf0f16-907a-431d-9e06-81cdf0155d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:19.054517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf0f16-907a-431d-9e06-81cdf0155d8b admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:19.059906 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88b4df-7a46-4c0a-a430-434381d214b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:19.060685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88b4df-7a46-4c0a-a430-434381d214b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:19.061047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88b4df-7a46-4c0a-a430-434381d214b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:19.061362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88b4df-7a46-4c0a-a430-434381d214b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:19.071143 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88b4df-7a46-4c0a-a430-434381d214b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:19.072659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88b4df-7a46-4c0a-a430-434381d214b5 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:19.084251 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1101/2208] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:19.104815 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aed2ae-07dc-4e07-9203-95ca69c90f96 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:19.108114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-10aed2ae-07dc-4e07-9203-95ca69c90f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:19.133062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aed2ae-07dc-4e07-9203-95ca69c90f96 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:19.142129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63645cc8-9aa2-4a82-99d4-0b53f6a5ab06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:19.142129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63645cc8-9aa2-4a82-99d4-0b53f6a5ab06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:19.142605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63645cc8-9aa2-4a82-99d4-0b53f6a5ab06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:19.142975 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63645cc8-9aa2-4a82-99d4-0b53f6a5ab06 admin admin] PATH_INFO: `/v3/users/d01f138c8f0d4940af6737d2576a7326` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:19.144564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63645cc8-9aa2-4a82-99d4-0b53f6a5ab06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d01f138c8f0d4940af6737d2576a7326)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:19.146954 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63645cc8-9aa2-4a82-99d4-0b53f6a5ab06 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:19.209422 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-63645cc8-9aa2-4a82-99d4-0b53f6a5ab06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd01f138c8f0d4940af6737d2576a7326'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:19.222323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-63645cc8-9aa2-4a82-99d4-0b53f6a5ab06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd01f138c8f0d4940af6737d2576a7326'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:19.229223 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-63645cc8-9aa2-4a82-99d4-0b53f6a5ab06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd01f138c8f0d4940af6737d2576a7326'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:19.229946 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-63645cc8-9aa2-4a82-99d4-0b53f6a5ab06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd01f138c8f0d4940af6737d2576a7326'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:19.230568 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-63645cc8-9aa2-4a82-99d4-0b53f6a5ab06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd01f138c8f0d4940af6737d2576a7326'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:19.246990 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1108/2209] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:19 2022] DELETE /identity/v3/users/d01f138c8f0d4940af6737d2576a7326 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:19.389090 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88b4df-7a46-4c0a-a430-434381d214b5 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:19.390621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3c88b4df-7a46-4c0a-a430-434381d214b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:19.404243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88b4df-7a46-4c0a-a430-434381d214b5 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:19.435655 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63645cc8-9aa2-4a82-99d4-0b53f6a5ab06 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:19.437083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-63645cc8-9aa2-4a82-99d4-0b53f6a5ab06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:19.484191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd75cf-dc2c-4244-bef0-7325b5acdedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:19.485221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd75cf-dc2c-4244-bef0-7325b5acdedf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:19.485590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd75cf-dc2c-4244-bef0-7325b5acdedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:19.485928 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd75cf-dc2c-4244-bef0-7325b5acdedf admin admin] PATH_INFO: `/v3/projects/7647815f455e44eca4a70921341c37df` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:19.487289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd75cf-dc2c-4244-bef0-7325b5acdedf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7647815f455e44eca4a70921341c37df)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:19.489532 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd75cf-dc2c-4244-bef0-7325b5acdedf admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:19.531470 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:19.537129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63645cc8-9aa2-4a82-99d4-0b53f6a5ab06 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:19.555444 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-88cd75cf-dc2c-4244-bef0-7325b5acdedf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7647815f455e44eca4a70921341c37df', 'request_id': 'req-88cd75cf-dc2c-4244-bef0-7325b5acdedf'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:19.564436 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-88cd75cf-dc2c-4244-bef0-7325b5acdedf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7647815f455e44eca4a70921341c37df', 'request_id': 'req-88cd75cf-dc2c-4244-bef0-7325b5acdedf'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:19.574100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-88cd75cf-dc2c-4244-bef0-7325b5acdedf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7647815f455e44eca4a70921341c37df', 'request_id': 'req-88cd75cf-dc2c-4244-bef0-7325b5acdedf'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:19.588950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:19.594854 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-88cd75cf-dc2c-4244-bef0-7325b5acdedf admin admin] The token cache is being invalidate because project 7647815f455e44eca4a70921341c37df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:19.595503 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-88cd75cf-dc2c-4244-bef0-7325b5acdedf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:19.597734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-88cd75cf-dc2c-4244-bef0-7325b5acdedf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:19.599624 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1102/2210] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:19 2022] DELETE /identity/v3/projects/7647815f455e44eca4a70921341c37df => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:19.606288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f34ca-17c0-4f48-a175-5d2a11fc8d26 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:19.607412 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f34ca-17c0-4f48-a175-5d2a11fc8d26 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:19.607768 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f34ca-17c0-4f48-a175-5d2a11fc8d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:19.631446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-577f34ca-17c0-4f48-a175-5d2a11fc8d26 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:19.632654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-577f34ca-17c0-4f48-a175-5d2a11fc8d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:19.633141 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018dec8-bce0-404d-a960-09d409e7efb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:19.634153 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018dec8-bce0-404d-a960-09d409e7efb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:19.634510 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018dec8-bce0-404d-a960-09d409e7efb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:19.634856 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018dec8-bce0-404d-a960-09d409e7efb8 admin admin] PATH_INFO: `/v3/projects/5a36ae0fd7b648158ce3fbe24d474ad7` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:19.636207 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018dec8-bce0-404d-a960-09d409e7efb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a36ae0fd7b648158ce3fbe24d474ad7)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:19.638130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018dec8-bce0-404d-a960-09d409e7efb8 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:19.641650 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1103/2211] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:45:19.649440 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2e439-baf3-49d4-b42b-52cedddaa79c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:19.649735 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2e439-baf3-49d4-b42b-52cedddaa79c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:19.650010 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2e439-baf3-49d4-b42b-52cedddaa79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:19.674108 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-9ce2e439-baf3-49d4-b42b-52cedddaa79c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:19.675367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9ce2e439-baf3-49d4-b42b-52cedddaa79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:19.683735 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1104/2212] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:19.688054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce2e439-baf3-49d4-b42b-52cedddaa79c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:19.697419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8018dec8-bce0-404d-a960-09d409e7efb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a36ae0fd7b648158ce3fbe24d474ad7', 'request_id': 'req-8018dec8-bce0-404d-a960-09d409e7efb8'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:19.697925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce2e439-baf3-49d4-b42b-52cedddaa79c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:19.705675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8018dec8-bce0-404d-a960-09d409e7efb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a36ae0fd7b648158ce3fbe24d474ad7', 'request_id': 'req-8018dec8-bce0-404d-a960-09d409e7efb8'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:19.715234 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8018dec8-bce0-404d-a960-09d409e7efb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a36ae0fd7b648158ce3fbe24d474ad7', 'request_id': 'req-8018dec8-bce0-404d-a960-09d409e7efb8'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:19.741252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8018dec8-bce0-404d-a960-09d409e7efb8 admin admin] The token cache is being invalidate because project 5a36ae0fd7b648158ce3fbe24d474ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:19.741681 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8018dec8-bce0-404d-a960-09d409e7efb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:19.742358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8018dec8-bce0-404d-a960-09d409e7efb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:19.744329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1109/2213] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:19 2022] DELETE /identity/v3/projects/5a36ae0fd7b648158ce3fbe24d474ad7 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:19.748934 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4ddff-c321-4ce7-973e-f5e0e9c97743 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:19.749345 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4ddff-c321-4ce7-973e-f5e0e9c97743 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:19.749727 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4ddff-c321-4ce7-973e-f5e0e9c97743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:19.775759 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050f699-71da-48a6-8b3f-226338297d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:19.775900 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-5db4ddff-c321-4ce7-973e-f5e0e9c97743 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:19.776501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050f699-71da-48a6-8b3f-226338297d41 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:19.776789 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050f699-71da-48a6-8b3f-226338297d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:19.776965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5db4ddff-c321-4ce7-973e-f5e0e9c97743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:19.777063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050f699-71da-48a6-8b3f-226338297d41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:19.778031 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050f699-71da-48a6-8b3f-226338297d41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:45:19.778474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050f699-71da-48a6-8b3f-226338297d41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:19.780520 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050f699-71da-48a6-8b3f-226338297d41 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:19.788214 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1110/2214] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:19.796448 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f52c6d-82c8-40c7-b09b-cf53c7a4c6e6 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:19.796583 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f52c6d-82c8-40c7-b09b-cf53c7a4c6e6 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:19.796995 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f52c6d-82c8-40c7-b09b-cf53c7a4c6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:19.801567 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1105/2215] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:45:19.806663 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050f699-71da-48a6-8b3f-226338297d41 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:19.808436 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-2050f699-71da-48a6-8b3f-226338297d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:19.819724 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-b2f52c6d-82c8-40c7-b09b-cf53c7a4c6e6 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:19.820973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b2f52c6d-82c8-40c7-b09b-cf53c7a4c6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:19.823114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050f699-71da-48a6-8b3f-226338297d41 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:19.828090 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075899e-7b50-4b68-b9ab-c8c706018f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:19.828799 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075899e-7b50-4b68-b9ab-c8c706018f23 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:19.829055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1111/2216] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:19.832229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075899e-7b50-4b68-b9ab-c8c706018f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:19.832229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075899e-7b50-4b68-b9ab-c8c706018f23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:19.832229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075899e-7b50-4b68-b9ab-c8c706018f23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:19.832229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075899e-7b50-4b68-b9ab-c8c706018f23 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:19.833142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f52c6d-82c8-40c7-b09b-cf53c7a4c6e6 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:19.841896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f52c6d-82c8-40c7-b09b-cf53c7a4c6e6 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:19.846524 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b81fc-330c-4087-a1a2-9c0fd217f3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:19.847243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b81fc-330c-4087-a1a2-9c0fd217f3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:19.847534 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b81fc-330c-4087-a1a2-9c0fd217f3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:19.847812 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b81fc-330c-4087-a1a2-9c0fd217f3c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:19.848579 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b81fc-330c-4087-a1a2-9c0fd217f3c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:45:19.848916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b81fc-330c-4087-a1a2-9c0fd217f3c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:19.850602 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b81fc-330c-4087-a1a2-9c0fd217f3c6 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:19.865227 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1106/2217] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:45:19 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:45:19.865387 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1112/2218] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:45:19.866250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075899e-7b50-4b68-b9ab-c8c706018f23 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:19.867930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b81fc-330c-4087-a1a2-9c0fd217f3c6 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:19.877453 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b81fc-330c-4087-a1a2-9c0fd217f3c6 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:19.929660 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075899e-7b50-4b68-b9ab-c8c706018f23 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:19.935274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3341e01-23ea-42c5-9cc0-526283d5e746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:19.935865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3341e01-23ea-42c5-9cc0-526283d5e746 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:19.936147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3341e01-23ea-42c5-9cc0-526283d5e746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:19.936442 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3341e01-23ea-42c5-9cc0-526283d5e746 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:19.937131 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3341e01-23ea-42c5-9cc0-526283d5e746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:19.937827 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cf1b5-0659-4743-ad70-afc63fbbd470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:19.938504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cf1b5-0659-4743-ad70-afc63fbbd470 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:19.938794 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cf1b5-0659-4743-ad70-afc63fbbd470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:19.939099 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3341e01-23ea-42c5-9cc0-526283d5e746 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:19.939198 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cf1b5-0659-4743-ad70-afc63fbbd470 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:19.939803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cf1b5-0659-4743-ad70-afc63fbbd470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:19.941728 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cf1b5-0659-4743-ad70-afc63fbbd470 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:19.956836 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:19.967724 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1107/2219] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:45:19 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:45:19.969016 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3341e01-23ea-42c5-9cc0-526283d5e746 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:19.976634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1113/2220] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:45:19 2022] POST /identity/v3/projects => generated 405 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:19.980167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3341e01-23ea-42c5-9cc0-526283d5e746 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:19.981754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cf1b5-0659-4743-ad70-afc63fbbd470 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:20.024234 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:20.039750 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19043c81-2bad-4886-8d1e-74bb6fd395df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:20.040441 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19043c81-2bad-4886-8d1e-74bb6fd395df admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:20.040732 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19043c81-2bad-4886-8d1e-74bb6fd395df admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:20.041019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19043c81-2bad-4886-8d1e-74bb6fd395df admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:20.041753 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19043c81-2bad-4886-8d1e-74bb6fd395df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:20.044145 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19043c81-2bad-4886-8d1e-74bb6fd395df admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:20.049926 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cf1b5-0659-4743-ad70-afc63fbbd470 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:20.054672 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bc697-6b2c-484a-8c37-49192cd1e3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:20.055429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1108/2221] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:45:20.055545 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bc697-6b2c-484a-8c37-49192cd1e3b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:20.055736 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bc697-6b2c-484a-8c37-49192cd1e3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:20.056112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74bc697-6b2c-484a-8c37-49192cd1e3b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:20.056817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74bc697-6b2c-484a-8c37-49192cd1e3b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:20.059078 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19043c81-2bad-4886-8d1e-74bb6fd395df admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:20.059188 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74bc697-6b2c-484a-8c37-49192cd1e3b5 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:20.068837 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19043c81-2bad-4886-8d1e-74bb6fd395df admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:20.074097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bdbdae-0ec7-4214-b11a-6510d0d7aedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:20.074756 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdbdae-0ec7-4214-b11a-6510d0d7aedc admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:20.075042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdbdae-0ec7-4214-b11a-6510d0d7aedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:20.075386 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdbdae-0ec7-4214-b11a-6510d0d7aedc admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:20.076112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bdbdae-0ec7-4214-b11a-6510d0d7aedc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:20.077753 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bdbdae-0ec7-4214-b11a-6510d0d7aedc admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:20.083535 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1114/2222] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:45:19 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:20.084529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bc697-6b2c-484a-8c37-49192cd1e3b5 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:20.088527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1109/2223] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:45:20.092716 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bdbdae-0ec7-4214-b11a-6510d0d7aedc admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:20.094452 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74bc697-6b2c-484a-8c37-49192cd1e3b5 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:20.099660 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b631d2-5c9f-4f5e-b136-7161176689e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:20.100362 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b631d2-5c9f-4f5e-b136-7161176689e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:20.100651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b631d2-5c9f-4f5e-b136-7161176689e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:20.100925 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b631d2-5c9f-4f5e-b136-7161176689e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:20.101699 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b631d2-5c9f-4f5e-b136-7161176689e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:20.102249 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bdbdae-0ec7-4214-b11a-6510d0d7aedc admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:20.103354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b631d2-5c9f-4f5e-b136-7161176689e2 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:20.106816 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a2175-7b7f-42e3-86d1-a9b69058a031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:20.107709 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a2175-7b7f-42e3-86d1-a9b69058a031 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:20.107997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a2175-7b7f-42e3-86d1-a9b69058a031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:20.108309 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a2175-7b7f-42e3-86d1-a9b69058a031 admin admin] PATH_INFO: `/v3/projects/b66c6ff9fade4d528d45545fc4806792/users/c7f987f5904d45a5bd6d3f991b9a406f/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:20.113951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1115/2224] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:45:20.117863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b631d2-5c9f-4f5e-b136-7161176689e2 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:20.123966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a2175-7b7f-42e3-86d1-a9b69058a031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b66c6ff9fade4d528d45545fc4806792, user_id=c7f987f5904d45a5bd6d3f991b9a406f, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:20.126713 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a2175-7b7f-42e3-86d1-a9b69058a031 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:20.127740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b631d2-5c9f-4f5e-b136-7161176689e2 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:20.133682 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb1546-c5e1-4d44-865d-9b4f9bb7bed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:20.134347 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb1546-c5e1-4d44-865d-9b4f9bb7bed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:20.134648 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb1546-c5e1-4d44-865d-9b4f9bb7bed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:20.134906 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb1546-c5e1-4d44-865d-9b4f9bb7bed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:20.135746 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb1546-c5e1-4d44-865d-9b4f9bb7bed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:20.137235 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb1546-c5e1-4d44-865d-9b4f9bb7bed6 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:20.145030 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1110/2225] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:20 2022] PUT /identity/v3/projects/b66c6ff9fade4d528d45545fc4806792/users/c7f987f5904d45a5bd6d3f991b9a406f/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:45:20.148962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1116/2226] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:20.153769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a2175-7b7f-42e3-86d1-a9b69058a031 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:20.156474 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7b5a5-a1bd-4226-befe-a9258ef1f6d1 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:20.156831 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7b5a5-a1bd-4226-befe-a9258ef1f6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:20.157172 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7b5a5-a1bd-4226-befe-a9258ef1f6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:20.199762 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-1dc7b5a5-a1bd-4226-befe-a9258ef1f6d1 None None] MFA Rules not processed for user `c7f987f5904d45a5bd6d3f991b9a406f`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:20.201507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1dc7b5a5-a1bd-4226-befe-a9258ef1f6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:20.227440 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a2175-7b7f-42e3-86d1-a9b69058a031 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:20.232762 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae7f91-3bc4-4216-b5ef-87f39308f7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:20.233770 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae7f91-3bc4-4216-b5ef-87f39308f7ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:20.234156 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae7f91-3bc4-4216-b5ef-87f39308f7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:20.234545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae7f91-3bc4-4216-b5ef-87f39308f7ee admin admin] PATH_INFO: `/v3/projects/cbfc092daed6427b9d95a7ffb1690174/users/07bccb2dece6437f9208cac2d06ff316/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:20.249540 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae7f91-3bc4-4216-b5ef-87f39308f7ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbfc092daed6427b9d95a7ffb1690174, user_id=07bccb2dece6437f9208cac2d06ff316, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:20.252344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae7f91-3bc4-4216-b5ef-87f39308f7ee admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:20.266489 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1111/2227] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:20 2022] PUT /identity/v3/projects/cbfc092daed6427b9d95a7ffb1690174/users/07bccb2dece6437f9208cac2d06ff316/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:20.273138 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1117/2228] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:20.276831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734394f-2cc4-4c54-8a74-20c992dce077 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:20.277267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734394f-2cc4-4c54-8a74-20c992dce077 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:20.277561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a734394f-2cc4-4c54-8a74-20c992dce077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:20.285011 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7b5a5-a1bd-4226-befe-a9258ef1f6d1 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:20.286412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1dc7b5a5-a1bd-4226-befe-a9258ef1f6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:20.303869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-a734394f-2cc4-4c54-8a74-20c992dce077 None None] MFA Rules not processed for user `07bccb2dece6437f9208cac2d06ff316`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:20.305069 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a734394f-2cc4-4c54-8a74-20c992dce077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:20.344281 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc7b5a5-a1bd-4226-befe-a9258ef1f6d1 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:20.348255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:20.374868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1112/2229] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:20 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:45:20.382265 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734394f-2cc4-4c54-8a74-20c992dce077 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:20.400333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa9249-ad8b-48be-a7bc-9386976b942c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:20.400994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa9249-ad8b-48be-a7bc-9386976b942c service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:20.401281 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa9249-ad8b-48be-a7bc-9386976b942c service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:20.401558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa9249-ad8b-48be-a7bc-9386976b942c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:20.407949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:20.446781 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a734394f-2cc4-4c54-8a74-20c992dce077 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:20.451730 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2e507-96c5-45d9-9169-cb153dc5b0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:20.452396 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2e507-96c5-45d9-9169-cb153dc5b0c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:20.452680 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2e507-96c5-45d9-9169-cb153dc5b0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:20.452959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2e507-96c5-45d9-9169-cb153dc5b0c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:20.453676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2e507-96c5-45d9-9169-cb153dc5b0c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:20.455697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2e507-96c5-45d9-9169-cb153dc5b0c4 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:20.458512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa9249-ad8b-48be-a7bc-9386976b942c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:20.460308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa9249-ad8b-48be-a7bc-9386976b942c service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:20.469273 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1118/2230] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:20 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:45:20.481501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1113/2231] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:45:20 2022] POST /identity/v3/projects => generated 403 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:20.485765 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa9249-ad8b-48be-a7bc-9386976b942c service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:20.496248 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa9249-ad8b-48be-a7bc-9386976b942c service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:20.557062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa333b7-f42b-4eb3-8e8e-034e5d9a9064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:20.557742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa333b7-f42b-4eb3-8e8e-034e5d9a9064 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:20.558024 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa333b7-f42b-4eb3-8e8e-034e5d9a9064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:20.558295 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa333b7-f42b-4eb3-8e8e-034e5d9a9064 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:20.558985 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa333b7-f42b-4eb3-8e8e-034e5d9a9064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:20.561274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa333b7-f42b-4eb3-8e8e-034e5d9a9064 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:20.585527 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1119/2232] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:45:20 2022] POST /identity/v3/users => generated 308 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:45:20.586795 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2e507-96c5-45d9-9169-cb153dc5b0c4 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:20.598533 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2e507-96c5-45d9-9169-cb153dc5b0c4 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:20.604240 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9a53c-3e10-43f4-a7c2-0f4ad2315758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:20.605130 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9a53c-3e10-43f4-a7c2-0f4ad2315758 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:20.605596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9a53c-3e10-43f4-a7c2-0f4ad2315758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:20.606026 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d9a53c-3e10-43f4-a7c2-0f4ad2315758 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:20.606924 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d9a53c-3e10-43f4-a7c2-0f4ad2315758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:20.608876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d9a53c-3e10-43f4-a7c2-0f4ad2315758 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:20.621306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1114/2233] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:20.627268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa333b7-f42b-4eb3-8e8e-034e5d9a9064 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:20.637851 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa333b7-f42b-4eb3-8e8e-034e5d9a9064 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:20.644357 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4617d-4e7f-4040-b489-48a0fdf70436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:20.644357 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4617d-4e7f-4040-b489-48a0fdf70436 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:20.644592 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4617d-4e7f-4040-b489-48a0fdf70436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:20.644996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4617d-4e7f-4040-b489-48a0fdf70436 admin admin] PATH_INFO: `/v3/projects/6082d357d8454c8c93db0047c4e23a59/users/79b16c36cc2b4b7bb909a9c5f2629491/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:20.669665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4617d-4e7f-4040-b489-48a0fdf70436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6082d357d8454c8c93db0047c4e23a59, user_id=79b16c36cc2b4b7bb909a9c5f2629491, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:20.672349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4617d-4e7f-4040-b489-48a0fdf70436 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:20.692079 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1120/2234] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:20 2022] PUT /identity/v3/projects/6082d357d8454c8c93db0047c4e23a59/users/79b16c36cc2b4b7bb909a9c5f2629491/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:20.695927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9a53c-3e10-43f4-a7c2-0f4ad2315758 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:20.706971 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d9a53c-3e10-43f4-a7c2-0f4ad2315758 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:20.781919 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac30fb3-1086-46d8-a7c8-71ff18463125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:20.782765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac30fb3-1086-46d8-a7c8-71ff18463125 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:20.785457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac30fb3-1086-46d8-a7c8-71ff18463125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:20.785835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac30fb3-1086-46d8-a7c8-71ff18463125 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:20.786799 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac30fb3-1086-46d8-a7c8-71ff18463125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:20.788711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac30fb3-1086-46d8-a7c8-71ff18463125 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:20.803487 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1115/2235] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:45:20.807406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4617d-4e7f-4040-b489-48a0fdf70436 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:20.819986 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4617d-4e7f-4040-b489-48a0fdf70436 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:20.825769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8c8fd-8aeb-4578-a693-c97c44d0254f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:20.827274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8c8fd-8aeb-4578-a693-c97c44d0254f admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:20.827652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8c8fd-8aeb-4578-a693-c97c44d0254f admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:20.828091 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8c8fd-8aeb-4578-a693-c97c44d0254f admin admin] PATH_INFO: `/v3/projects/6082d357d8454c8c93db0047c4e23a59/users/79b16c36cc2b4b7bb909a9c5f2629491/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:20.831498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8c8fd-8aeb-4578-a693-c97c44d0254f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6082d357d8454c8c93db0047c4e23a59, user_id=79b16c36cc2b4b7bb909a9c5f2629491, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:20.834490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8c8fd-8aeb-4578-a693-c97c44d0254f admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:20.859137 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1121/2236] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:20 2022] PUT /identity/v3/projects/6082d357d8454c8c93db0047c4e23a59/users/79b16c36cc2b4b7bb909a9c5f2629491/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:45:20.869356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833a229-e2e3-4d3b-8a07-4965a1d079ec None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:20.869836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833a229-e2e3-4d3b-8a07-4965a1d079ec None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:20.869924 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833a229-e2e3-4d3b-8a07-4965a1d079ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:20.896301 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-a833a229-e2e3-4d3b-8a07-4965a1d079ec None None] MFA Rules not processed for user `79b16c36cc2b4b7bb909a9c5f2629491`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:20.897514 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a833a229-e2e3-4d3b-8a07-4965a1d079ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:20.970581 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1116/2237] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:20 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:20.982983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8c8fd-8aeb-4578-a693-c97c44d0254f admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:20.984874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-39d8c8fd-8aeb-4578-a693-c97c44d0254f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:21.000092 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:21.051501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8c8fd-8aeb-4578-a693-c97c44d0254f admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:21.057337 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd2be6-1e58-4cf8-95d4-905d792a9ced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:21.058260 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd2be6-1e58-4cf8-95d4-905d792a9ced service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:21.058608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd2be6-1e58-4cf8-95d4-905d792a9ced service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:21.058961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd2be6-1e58-4cf8-95d4-905d792a9ced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:21.070309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd2be6-1e58-4cf8-95d4-905d792a9ced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:21.071798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd2be6-1e58-4cf8-95d4-905d792a9ced service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:21.081381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1122/2238] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:20 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:21.263705 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833a229-e2e3-4d3b-8a07-4965a1d079ec None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:21.286559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:21.332635 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833a229-e2e3-4d3b-8a07-4965a1d079ec None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:21.337330 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea273f0-3d4f-4201-8a20-acc6f642ed60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:21.338218 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea273f0-3d4f-4201-8a20-acc6f642ed60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:21.338469 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea273f0-3d4f-4201-8a20-acc6f642ed60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:21.338738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea273f0-3d4f-4201-8a20-acc6f642ed60 admin admin] PATH_INFO: `/v3/users/07bccb2dece6437f9208cac2d06ff316` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:21.340002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea273f0-3d4f-4201-8a20-acc6f642ed60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07bccb2dece6437f9208cac2d06ff316)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:21.341933 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea273f0-3d4f-4201-8a20-acc6f642ed60 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:21.395266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-aea273f0-3d4f-4201-8a20-acc6f642ed60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07bccb2dece6437f9208cac2d06ff316'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:21.407332 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-aea273f0-3d4f-4201-8a20-acc6f642ed60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07bccb2dece6437f9208cac2d06ff316'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:21.414389 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-aea273f0-3d4f-4201-8a20-acc6f642ed60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07bccb2dece6437f9208cac2d06ff316'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:21.415082 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-aea273f0-3d4f-4201-8a20-acc6f642ed60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07bccb2dece6437f9208cac2d06ff316'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:21.415735 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-aea273f0-3d4f-4201-8a20-acc6f642ed60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07bccb2dece6437f9208cac2d06ff316'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:21.430451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1117/2239] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:21 2022] DELETE /identity/v3/users/07bccb2dece6437f9208cac2d06ff316 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:21.435389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb406c8-b6d2-410a-add1-ddd9c76cdd54 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:21.435686 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb406c8-b6d2-410a-add1-ddd9c76cdd54 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:21.435967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb406c8-b6d2-410a-add1-ddd9c76cdd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:21.453397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-bbb406c8-b6d2-410a-add1-ddd9c76cdd54 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:21.454586 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-bbb406c8-b6d2-410a-add1-ddd9c76cdd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:21.525458 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1123/2240] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:21.535872 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea273f0-3d4f-4201-8a20-acc6f642ed60 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:21.537326 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-aea273f0-3d4f-4201-8a20-acc6f642ed60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:21.595986 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea273f0-3d4f-4201-8a20-acc6f642ed60 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:21.601010 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd5767-7b46-4270-8ee5-e3c8c493df3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:21.601656 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd5767-7b46-4270-8ee5-e3c8c493df3b service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:21.601938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd5767-7b46-4270-8ee5-e3c8c493df3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:21.602208 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd5767-7b46-4270-8ee5-e3c8c493df3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:21.612753 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd5767-7b46-4270-8ee5-e3c8c493df3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:21.614273 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd5767-7b46-4270-8ee5-e3c8c493df3b service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:21.623856 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1118/2241] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:21.903106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb406c8-b6d2-410a-add1-ddd9c76cdd54 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:21.905369 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-bbb406c8-b6d2-410a-add1-ddd9c76cdd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:21.927326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb406c8-b6d2-410a-add1-ddd9c76cdd54 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:21.932743 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:21.933601 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:21.933944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:21.934280 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb admin admin] PATH_INFO: `/v3/projects/cbfc092daed6427b9d95a7ffb1690174` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:21.935572 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbfc092daed6427b9d95a7ffb1690174)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:21.938083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:21.993092 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbfc092daed6427b9d95a7ffb1690174', 'request_id': 'req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:21.999052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbfc092daed6427b9d95a7ffb1690174', 'request_id': 'req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:22.007697 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbfc092daed6427b9d95a7ffb1690174', 'request_id': 'req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:22.019534 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb admin admin] The token cache is being invalidate because project cbfc092daed6427b9d95a7ffb1690174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:22.019834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:22.020531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:22.022276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1124/2242] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:21 2022] DELETE /identity/v3/projects/cbfc092daed6427b9d95a7ffb1690174 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:22.026487 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd5767-7b46-4270-8ee5-e3c8c493df3b service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:22.036983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd5767-7b46-4270-8ee5-e3c8c493df3b service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:22.074887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:22.096774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a409f-80ca-4c52-9e51-9e0e29e24a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:22.097654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a409f-80ca-4c52-9e51-9e0e29e24a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:22.097932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a409f-80ca-4c52-9e51-9e0e29e24a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:22.098205 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a409f-80ca-4c52-9e51-9e0e29e24a6b admin admin] PATH_INFO: `/v3/users/79b16c36cc2b4b7bb909a9c5f2629491` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:22.099414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a409f-80ca-4c52-9e51-9e0e29e24a6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=79b16c36cc2b4b7bb909a9c5f2629491)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:22.101350 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a409f-80ca-4c52-9e51-9e0e29e24a6b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:22.151414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-883a409f-80ca-4c52-9e51-9e0e29e24a6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79b16c36cc2b4b7bb909a9c5f2629491'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:22.163349 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-883a409f-80ca-4c52-9e51-9e0e29e24a6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79b16c36cc2b4b7bb909a9c5f2629491'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:22.170174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-883a409f-80ca-4c52-9e51-9e0e29e24a6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79b16c36cc2b4b7bb909a9c5f2629491'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:22.170869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-883a409f-80ca-4c52-9e51-9e0e29e24a6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79b16c36cc2b4b7bb909a9c5f2629491'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:22.171533 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-883a409f-80ca-4c52-9e51-9e0e29e24a6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79b16c36cc2b4b7bb909a9c5f2629491'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:22.185701 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1119/2243] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:22 2022] DELETE /identity/v3/users/79b16c36cc2b4b7bb909a9c5f2629491 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:22.489169 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:22.490666 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:22.505733 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18e6bb-bf2e-4160-9b82-f8b23d28a8cb admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:22.547650 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:22.567338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972bd6c-9067-4b43-9615-10baca81b4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:22.568215 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972bd6c-9067-4b43-9615-10baca81b4f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:22.568505 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972bd6c-9067-4b43-9615-10baca81b4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:22.568775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972bd6c-9067-4b43-9615-10baca81b4f1 admin admin] PATH_INFO: `/v3/projects/6082d357d8454c8c93db0047c4e23a59` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:22.569957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972bd6c-9067-4b43-9615-10baca81b4f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6082d357d8454c8c93db0047c4e23a59)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:22.571928 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972bd6c-9067-4b43-9615-10baca81b4f1 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:22.619193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c972bd6c-9067-4b43-9615-10baca81b4f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6082d357d8454c8c93db0047c4e23a59', 'request_id': 'req-c972bd6c-9067-4b43-9615-10baca81b4f1'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:22.624939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c972bd6c-9067-4b43-9615-10baca81b4f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6082d357d8454c8c93db0047c4e23a59', 'request_id': 'req-c972bd6c-9067-4b43-9615-10baca81b4f1'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:22.633104 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c972bd6c-9067-4b43-9615-10baca81b4f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6082d357d8454c8c93db0047c4e23a59', 'request_id': 'req-c972bd6c-9067-4b43-9615-10baca81b4f1'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:22.644121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c972bd6c-9067-4b43-9615-10baca81b4f1 admin admin] The token cache is being invalidate because project 6082d357d8454c8c93db0047c4e23a59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:22.644414 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c972bd6c-9067-4b43-9615-10baca81b4f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:22.645043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c972bd6c-9067-4b43-9615-10baca81b4f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:22.646662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1125/2244] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:22 2022] DELETE /identity/v3/projects/6082d357d8454c8c93db0047c4e23a59 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:22.651892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c6ee6-3e93-49e1-bc49-b7996e7697c3 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:22.652171 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c6ee6-3e93-49e1-bc49-b7996e7697c3 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:22.652415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c6ee6-3e93-49e1-bc49-b7996e7697c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:22.670885 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-dc2c6ee6-3e93-49e1-bc49-b7996e7697c3 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:22.672166 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-dc2c6ee6-3e93-49e1-bc49-b7996e7697c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:22.733917 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1120/2245] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:22.746959 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e5f7c-f44a-4374-894a-9f4c17f3fa9a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:22.747280 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e5f7c-f44a-4374-894a-9f4c17f3fa9a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:22.747567 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e5f7c-f44a-4374-894a-9f4c17f3fa9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:22.766245 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-f05e5f7c-f44a-4374-894a-9f4c17f3fa9a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:22.767494 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f05e5f7c-f44a-4374-894a-9f4c17f3fa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:22.776129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1126/2246] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:22.780482 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c6ee6-3e93-49e1-bc49-b7996e7697c3 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:22.791322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c6ee6-3e93-49e1-bc49-b7996e7697c3 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:22.796446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151adc9d-b600-4f11-9021-39563ef37609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:22.797096 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151adc9d-b600-4f11-9021-39563ef37609 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:22.797375 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151adc9d-b600-4f11-9021-39563ef37609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:22.797731 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151adc9d-b600-4f11-9021-39563ef37609 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:22.798398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151adc9d-b600-4f11-9021-39563ef37609 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:45:22.798703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151adc9d-b600-4f11-9021-39563ef37609 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:22.800363 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151adc9d-b600-4f11-9021-39563ef37609 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:22.812965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1121/2247] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:45:22.817422 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e5f7c-f44a-4374-894a-9f4c17f3fa9a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:22.826896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e5f7c-f44a-4374-894a-9f4c17f3fa9a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:22.832170 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced78795-854d-449d-9801-d11dd3320820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:22.832820 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced78795-854d-449d-9801-d11dd3320820 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:22.833101 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced78795-854d-449d-9801-d11dd3320820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:22.833377 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced78795-854d-449d-9801-d11dd3320820 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:22.834069 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced78795-854d-449d-9801-d11dd3320820 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:22.836112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced78795-854d-449d-9801-d11dd3320820 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:22.865745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1127/2248] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:45:22 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:22.869983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151adc9d-b600-4f11-9021-39563ef37609 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:22.880072 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151adc9d-b600-4f11-9021-39563ef37609 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:22.924752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:22.941931 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3d77a-d544-4ebf-8a3e-e6e84746a9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:22.942614 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3d77a-d544-4ebf-8a3e-e6e84746a9ab admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:22.942895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3d77a-d544-4ebf-8a3e-e6e84746a9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:22.943205 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3d77a-d544-4ebf-8a3e-e6e84746a9ab admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:22.943940 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3d77a-d544-4ebf-8a3e-e6e84746a9ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:22.945887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3d77a-d544-4ebf-8a3e-e6e84746a9ab admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:22.969186 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1122/2249] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:45:22 2022] POST /identity/v3/users => generated 302 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:22.973350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced78795-854d-449d-9801-d11dd3320820 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:22.983921 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced78795-854d-449d-9801-d11dd3320820 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:22.988869 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce7075-7fc2-443e-a943-a05aef15e157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:22.989537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce7075-7fc2-443e-a943-a05aef15e157 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:22.989822 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce7075-7fc2-443e-a943-a05aef15e157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:22.990098 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce7075-7fc2-443e-a943-a05aef15e157 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:22.990838 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce7075-7fc2-443e-a943-a05aef15e157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:22.992510 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce7075-7fc2-443e-a943-a05aef15e157 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:23.003664 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1128/2250] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:23.007676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3d77a-d544-4ebf-8a3e-e6e84746a9ab admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:23.018469 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3d77a-d544-4ebf-8a3e-e6e84746a9ab admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:23.024418 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961f198-c567-4204-b7d3-db0797fab98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:23.025102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961f198-c567-4204-b7d3-db0797fab98c admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:23.025388 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961f198-c567-4204-b7d3-db0797fab98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:23.025663 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961f198-c567-4204-b7d3-db0797fab98c admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:23.026392 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961f198-c567-4204-b7d3-db0797fab98c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:23.028029 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961f198-c567-4204-b7d3-db0797fab98c admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:23.038960 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1123/2251] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:23.043288 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce7075-7fc2-443e-a943-a05aef15e157 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:23.054504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce7075-7fc2-443e-a943-a05aef15e157 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:23.059960 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848cfc0-c404-4d1a-aa98-ae283e7434bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:23.060898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848cfc0-c404-4d1a-aa98-ae283e7434bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:23.061185 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848cfc0-c404-4d1a-aa98-ae283e7434bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:23.061455 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848cfc0-c404-4d1a-aa98-ae283e7434bd admin admin] PATH_INFO: `/v3/projects/950f370f559e4e68ae0da445a64312ec/users/8ca5e09076ef458f90b745da11d8fd8f/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:23.077237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848cfc0-c404-4d1a-aa98-ae283e7434bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=950f370f559e4e68ae0da445a64312ec, user_id=8ca5e09076ef458f90b745da11d8fd8f, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:23.079931 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848cfc0-c404-4d1a-aa98-ae283e7434bd admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:23.094388 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1129/2252] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:23 2022] PUT /identity/v3/projects/950f370f559e4e68ae0da445a64312ec/users/8ca5e09076ef458f90b745da11d8fd8f/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:23.099892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df648a04-bf1f-4f52-981b-ad3469da3dab None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:23.100190 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df648a04-bf1f-4f52-981b-ad3469da3dab None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:23.100466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df648a04-bf1f-4f52-981b-ad3469da3dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:23.119722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-df648a04-bf1f-4f52-981b-ad3469da3dab None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:23.121065 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-df648a04-bf1f-4f52-981b-ad3469da3dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:23.184575 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1124/2253] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:23.194739 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848cfc0-c404-4d1a-aa98-ae283e7434bd admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:23.196605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-7848cfc0-c404-4d1a-aa98-ae283e7434bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:23.253918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848cfc0-c404-4d1a-aa98-ae283e7434bd admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:23.257542 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8e871-5431-4949-adde-e967656ddfc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:23.258182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8e871-5431-4949-adde-e967656ddfc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:23.258467 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8e871-5431-4949-adde-e967656ddfc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:23.258745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8e871-5431-4949-adde-e967656ddfc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:23.267697 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8e871-5431-4949-adde-e967656ddfc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:23.269200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8e871-5431-4949-adde-e967656ddfc5 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:23.278395 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1130/2254] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:23.894396 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20508055-d143-43db-a3cd-29713dce0942 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:23.894690 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20508055-d143-43db-a3cd-29713dce0942 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:23.894970 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20508055-d143-43db-a3cd-29713dce0942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:23.923760 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-20508055-d143-43db-a3cd-29713dce0942 None None] MFA Rules not processed for user `8ca5e09076ef458f90b745da11d8fd8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:23.925268 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-20508055-d143-43db-a3cd-29713dce0942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:24.001774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1125/2255] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:24.006314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8e871-5431-4949-adde-e967656ddfc5 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:24.024117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8e871-5431-4949-adde-e967656ddfc5 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:24.024117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e00b8a-aebd-4fa8-960b-1796b4bc6f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:24.024117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e00b8a-aebd-4fa8-960b-1796b4bc6f78 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:24.024117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e00b8a-aebd-4fa8-960b-1796b4bc6f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:24.024492 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e00b8a-aebd-4fa8-960b-1796b4bc6f78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:24.024492 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e00b8a-aebd-4fa8-960b-1796b4bc6f78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:24.024877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e00b8a-aebd-4fa8-960b-1796b4bc6f78 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:24.029043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:24.057063 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1131/2256] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:45:24 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:45:24.058526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20508055-d143-43db-a3cd-29713dce0942 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:24.069874 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20508055-d143-43db-a3cd-29713dce0942 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:24.120963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:24.129759 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d376e8b-01c8-46c8-b5ae-e9e234dd0842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:24.130448 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d376e8b-01c8-46c8-b5ae-e9e234dd0842 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:24.130733 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d376e8b-01c8-46c8-b5ae-e9e234dd0842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:24.131032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d376e8b-01c8-46c8-b5ae-e9e234dd0842 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:24.131790 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d376e8b-01c8-46c8-b5ae-e9e234dd0842 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:24.134091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d376e8b-01c8-46c8-b5ae-e9e234dd0842 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:24.156343 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1126/2257] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:45:24 2022] POST /identity/v3/users => generated 309 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:24.161705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e00b8a-aebd-4fa8-960b-1796b4bc6f78 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:24.172607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e00b8a-aebd-4fa8-960b-1796b4bc6f78 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:24.178011 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a331d-fd6b-432b-bff2-9dcf3d056676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:24.178678 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a331d-fd6b-432b-bff2-9dcf3d056676 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:24.178962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a331d-fd6b-432b-bff2-9dcf3d056676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:24.179271 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a331d-fd6b-432b-bff2-9dcf3d056676 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:24.180001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a331d-fd6b-432b-bff2-9dcf3d056676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:24.181622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a331d-fd6b-432b-bff2-9dcf3d056676 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:24.192995 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1132/2258] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:24.197863 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d376e8b-01c8-46c8-b5ae-e9e234dd0842 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:24.212904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d376e8b-01c8-46c8-b5ae-e9e234dd0842 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:24.221519 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92b550-8fff-49fb-bc47-b39f8491d995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:24.222414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92b550-8fff-49fb-bc47-b39f8491d995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:24.222684 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92b550-8fff-49fb-bc47-b39f8491d995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:24.222961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92b550-8fff-49fb-bc47-b39f8491d995 admin admin] PATH_INFO: `/v3/projects/b10e877cd8eb4f2f948db4ac229f85d5/users/4d872d227f0f4a0d8a36397dcb5f32b0/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:24.238981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92b550-8fff-49fb-bc47-b39f8491d995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10e877cd8eb4f2f948db4ac229f85d5, user_id=4d872d227f0f4a0d8a36397dcb5f32b0, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:24.241689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92b550-8fff-49fb-bc47-b39f8491d995 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:24.255572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1127/2259] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:24 2022] PUT /identity/v3/projects/b10e877cd8eb4f2f948db4ac229f85d5/users/4d872d227f0f4a0d8a36397dcb5f32b0/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:24.259712 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a331d-fd6b-432b-bff2-9dcf3d056676 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:24.269415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a331d-fd6b-432b-bff2-9dcf3d056676 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:24.329598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed8c28-c80b-49d0-be37-4a3a61a1bd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:24.330276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed8c28-c80b-49d0-be37-4a3a61a1bd45 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:24.330587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed8c28-c80b-49d0-be37-4a3a61a1bd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:24.330862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed8c28-c80b-49d0-be37-4a3a61a1bd45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:24.331629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed8c28-c80b-49d0-be37-4a3a61a1bd45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:24.333219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed8c28-c80b-49d0-be37-4a3a61a1bd45 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:24.343779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1133/2260] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:24.348083 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92b550-8fff-49fb-bc47-b39f8491d995 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:24.358296 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92b550-8fff-49fb-bc47-b39f8491d995 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:24.363380 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e290a-0707-435c-b394-0ec00b352778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:24.364298 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e290a-0707-435c-b394-0ec00b352778 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:24.364611 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e290a-0707-435c-b394-0ec00b352778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:24.364888 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e290a-0707-435c-b394-0ec00b352778 admin admin] PATH_INFO: `/v3/projects/b10e877cd8eb4f2f948db4ac229f85d5/users/4d872d227f0f4a0d8a36397dcb5f32b0/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:24.366946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e290a-0707-435c-b394-0ec00b352778 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10e877cd8eb4f2f948db4ac229f85d5, user_id=4d872d227f0f4a0d8a36397dcb5f32b0, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:24.369741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e290a-0707-435c-b394-0ec00b352778 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:24.381813 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1128/2261] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:24 2022] PUT /identity/v3/projects/b10e877cd8eb4f2f948db4ac229f85d5/users/4d872d227f0f4a0d8a36397dcb5f32b0/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:45:25.077358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471327b9-a727-4297-a0e3-9c45d976f38f None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:25.077907 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471327b9-a727-4297-a0e3-9c45d976f38f None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:25.078009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471327b9-a727-4297-a0e3-9c45d976f38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:25.105991 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-471327b9-a727-4297-a0e3-9c45d976f38f None None] MFA Rules not processed for user `4d872d227f0f4a0d8a36397dcb5f32b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:25.107243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-471327b9-a727-4297-a0e3-9c45d976f38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:25.179590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1134/2262] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:25 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:25.749696 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f50021-7c37-4047-8a8a-7578f805cfe3 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:25.750221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f50021-7c37-4047-8a8a-7578f805cfe3 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:25.750371 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f50021-7c37-4047-8a8a-7578f805cfe3 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:25.751839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1129/2263] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jul 21 05:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:45:25.762794 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02e445-70c4-4093-b83d-21eb60594d86 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:25.763121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02e445-70c4-4093-b83d-21eb60594d86 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:25.763433 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02e445-70c4-4093-b83d-21eb60594d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:25.781767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-ad02e445-70c4-4093-b83d-21eb60594d86 None None] MFA Rules not processed for user `03d868598e39448881b43d299253d821`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:25.783100 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ad02e445-70c4-4093-b83d-21eb60594d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:25.840675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:25.865905 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1135/2264] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jul 21 05:45:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:25.880480 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f50021-7c37-4047-8a8a-7578f805cfe3 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:25.882264 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-34f50021-7c37-4047-8a8a-7578f805cfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:25.899947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f50021-7c37-4047-8a8a-7578f805cfe3 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:25.906869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb17ba8-9e1d-49f7-8fe8-0b8f9678d61f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:25.907650 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb17ba8-9e1d-49f7-8fe8-0b8f9678d61f service placement] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:25.907998 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb17ba8-9e1d-49f7-8fe8-0b8f9678d61f service placement] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:25.908368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb17ba8-9e1d-49f7-8fe8-0b8f9678d61f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:25.921969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb17ba8-9e1d-49f7-8fe8-0b8f9678d61f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:25.923594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb17ba8-9e1d-49f7-8fe8-0b8f9678d61f service placement] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:25.934321 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1130/2265] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:26.175527 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02e445-70c4-4093-b83d-21eb60594d86 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:26.177026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ad02e445-70c4-4093-b83d-21eb60594d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:26.254778 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02e445-70c4-4093-b83d-21eb60594d86 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:26.260111 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e42d6-e963-48c9-ae95-1aa9380bf779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:26.261332 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e42d6-e963-48c9-ae95-1aa9380bf779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:26.261665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e42d6-e963-48c9-ae95-1aa9380bf779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:26.261914 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e42d6-e963-48c9-ae95-1aa9380bf779 admin admin] PATH_INFO: `/v3/users/fac690a92b314d878e873e82cd0bb894` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:26.263271 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e42d6-e963-48c9-ae95-1aa9380bf779 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac690a92b314d878e873e82cd0bb894)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:26.265530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e42d6-e963-48c9-ae95-1aa9380bf779 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:26.340087 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0c0e42d6-e963-48c9-ae95-1aa9380bf779 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac690a92b314d878e873e82cd0bb894'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:26.353075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0c0e42d6-e963-48c9-ae95-1aa9380bf779 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac690a92b314d878e873e82cd0bb894'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:26.361012 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0c0e42d6-e963-48c9-ae95-1aa9380bf779 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac690a92b314d878e873e82cd0bb894'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:26.361839 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0c0e42d6-e963-48c9-ae95-1aa9380bf779 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac690a92b314d878e873e82cd0bb894'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:26.362930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0c0e42d6-e963-48c9-ae95-1aa9380bf779 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac690a92b314d878e873e82cd0bb894'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:26.380653 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1136/2266] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:26 2022] DELETE /identity/v3/users/fac690a92b314d878e873e82cd0bb894 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:26.385313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456420a0-e72a-4cb2-851a-ba8042530197 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:26.385605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456420a0-e72a-4cb2-851a-ba8042530197 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:26.385883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456420a0-e72a-4cb2-851a-ba8042530197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:26.404786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-456420a0-e72a-4cb2-851a-ba8042530197 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:26.406530 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-456420a0-e72a-4cb2-851a-ba8042530197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:26.443822 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e42d6-e963-48c9-ae95-1aa9380bf779 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:26.445669 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-0c0e42d6-e963-48c9-ae95-1aa9380bf779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:26.489556 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1131/2267] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:26.496895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456420a0-e72a-4cb2-851a-ba8042530197 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:26.498500 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-456420a0-e72a-4cb2-851a-ba8042530197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:26.531160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e42d6-e963-48c9-ae95-1aa9380bf779 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:26.539690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cf56a-181d-41c0-b4b6-fe7078c24c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:26.540818 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cf56a-181d-41c0-b4b6-fe7078c24c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:26.541115 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cf56a-181d-41c0-b4b6-fe7078c24c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:26.541441 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0cf56a-181d-41c0-b4b6-fe7078c24c52 admin admin] PATH_INFO: `/v3/users/8ca5e09076ef458f90b745da11d8fd8f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:26.542916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0cf56a-181d-41c0-b4b6-fe7078c24c52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ca5e09076ef458f90b745da11d8fd8f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:26.545050 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0cf56a-181d-41c0-b4b6-fe7078c24c52 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:26.568781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:26.589475 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456420a0-e72a-4cb2-851a-ba8042530197 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:26.596623 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1bfee-82d8-4ec2-a64c-f4a00b72c80a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:26.597546 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1bfee-82d8-4ec2-a64c-f4a00b72c80a service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:26.598302 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1bfee-82d8-4ec2-a64c-f4a00b72c80a service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:26.598302 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1bfee-82d8-4ec2-a64c-f4a00b72c80a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:26.610796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1bfee-82d8-4ec2-a64c-f4a00b72c80a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:26.613742 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1bfee-82d8-4ec2-a64c-f4a00b72c80a service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:26.626260 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-df0cf56a-181d-41c0-b4b6-fe7078c24c52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ca5e09076ef458f90b745da11d8fd8f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:26.639164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1132/2268] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:45:26.664133 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-df0cf56a-181d-41c0-b4b6-fe7078c24c52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ca5e09076ef458f90b745da11d8fd8f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:26.680145 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-df0cf56a-181d-41c0-b4b6-fe7078c24c52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ca5e09076ef458f90b745da11d8fd8f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:26.680920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-df0cf56a-181d-41c0-b4b6-fe7078c24c52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ca5e09076ef458f90b745da11d8fd8f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:26.681552 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-df0cf56a-181d-41c0-b4b6-fe7078c24c52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ca5e09076ef458f90b745da11d8fd8f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:26.700443 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1137/2269] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:26 2022] DELETE /identity/v3/users/8ca5e09076ef458f90b745da11d8fd8f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:26.897343 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1bfee-82d8-4ec2-a64c-f4a00b72c80a service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:26.917127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1bfee-82d8-4ec2-a64c-f4a00b72c80a service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:26.956738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cf56a-181d-41c0-b4b6-fe7078c24c52 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:26.958430 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-df0cf56a-181d-41c0-b4b6-fe7078c24c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:26.964034 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:26.996530 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:26.997421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:26.997703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:26.997971 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20 admin admin] PATH_INFO: `/v3/projects/950f370f559e4e68ae0da445a64312ec` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:26.999279 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=950f370f559e4e68ae0da445a64312ec)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:27.001284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:27.075808 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '950f370f559e4e68ae0da445a64312ec', 'request_id': 'req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.082572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '950f370f559e4e68ae0da445a64312ec', 'request_id': 'req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.090500 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0cf56a-181d-41c0-b4b6-fe7078c24c52 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:27.091418 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '950f370f559e4e68ae0da445a64312ec', 'request_id': 'req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.108829 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20 admin admin] The token cache is being invalidate because project 950f370f559e4e68ae0da445a64312ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:27.109608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.110441 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.114691 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1133/2270] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:26 2022] DELETE /identity/v3/projects/950f370f559e4e68ae0da445a64312ec => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:27.116280 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:27.117743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:27.168114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64675c6c-3bb7-4d14-9779-17bb01d6fed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:27.169368 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64675c6c-3bb7-4d14-9779-17bb01d6fed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:27.169880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64675c6c-3bb7-4d14-9779-17bb01d6fed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:27.170974 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64675c6c-3bb7-4d14-9779-17bb01d6fed9 admin admin] PATH_INFO: `/v3/projects/7845b6b63db14fc9be25dbc720b08eda` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:27.172370 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64675c6c-3bb7-4d14-9779-17bb01d6fed9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7845b6b63db14fc9be25dbc720b08eda)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:27.174391 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64675c6c-3bb7-4d14-9779-17bb01d6fed9 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:27.186006 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fa546-bd82-4e7b-ab16-0746e68ecf20 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:27.191063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687816e-7d38-42d9-a1b4-140cd15914e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:27.192099 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687816e-7d38-42d9-a1b4-140cd15914e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:27.192452 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687816e-7d38-42d9-a1b4-140cd15914e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:27.192804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687816e-7d38-42d9-a1b4-140cd15914e6 admin admin] PATH_INFO: `/v3/users/4d872d227f0f4a0d8a36397dcb5f32b0` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:27.194071 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687816e-7d38-42d9-a1b4-140cd15914e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d872d227f0f4a0d8a36397dcb5f32b0)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:27.196162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687816e-7d38-42d9-a1b4-140cd15914e6 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:27.223140 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:27.231628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-64675c6c-3bb7-4d14-9779-17bb01d6fed9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7845b6b63db14fc9be25dbc720b08eda', 'request_id': 'req-64675c6c-3bb7-4d14-9779-17bb01d6fed9'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.240231 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-64675c6c-3bb7-4d14-9779-17bb01d6fed9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7845b6b63db14fc9be25dbc720b08eda', 'request_id': 'req-64675c6c-3bb7-4d14-9779-17bb01d6fed9'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.247822 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-64675c6c-3bb7-4d14-9779-17bb01d6fed9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7845b6b63db14fc9be25dbc720b08eda', 'request_id': 'req-64675c6c-3bb7-4d14-9779-17bb01d6fed9'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.249290 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-4687816e-7d38-42d9-a1b4-140cd15914e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d872d227f0f4a0d8a36397dcb5f32b0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.259389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-64675c6c-3bb7-4d14-9779-17bb01d6fed9 admin admin] The token cache is being invalidate because project 7845b6b63db14fc9be25dbc720b08eda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:27.259754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-64675c6c-3bb7-4d14-9779-17bb01d6fed9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.260458 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-64675c6c-3bb7-4d14-9779-17bb01d6fed9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.261117 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-4687816e-7d38-42d9-a1b4-140cd15914e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d872d227f0f4a0d8a36397dcb5f32b0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.262270 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1138/2271] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:26 2022] DELETE /identity/v3/projects/7845b6b63db14fc9be25dbc720b08eda => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:27.266152 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64675c6c-3bb7-4d14-9779-17bb01d6fed9 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:27.266763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:27.267963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-64675c6c-3bb7-4d14-9779-17bb01d6fed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:27.269447 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-4687816e-7d38-42d9-a1b4-140cd15914e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d872d227f0f4a0d8a36397dcb5f32b0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.270204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-4687816e-7d38-42d9-a1b4-140cd15914e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d872d227f0f4a0d8a36397dcb5f32b0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.271099 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-4687816e-7d38-42d9-a1b4-140cd15914e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d872d227f0f4a0d8a36397dcb5f32b0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.286186 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1134/2272] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:27 2022] DELETE /identity/v3/users/4d872d227f0f4a0d8a36397dcb5f32b0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:45:27.336278 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64675c6c-3bb7-4d14-9779-17bb01d6fed9 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:27.396454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70a4d5-a2a5-4651-803a-d8686348d914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:27.397379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70a4d5-a2a5-4651-803a-d8686348d914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:27.397668 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70a4d5-a2a5-4651-803a-d8686348d914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:27.397943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70a4d5-a2a5-4651-803a-d8686348d914 admin admin] PATH_INFO: `/v3/users/ba0744a7da2c41c194bbbe8774bbaeb7` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:27.399518 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70a4d5-a2a5-4651-803a-d8686348d914 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba0744a7da2c41c194bbbe8774bbaeb7)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:27.402392 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70a4d5-a2a5-4651-803a-d8686348d914 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:27.455703 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5b70a4d5-a2a5-4651-803a-d8686348d914 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba0744a7da2c41c194bbbe8774bbaeb7'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.463349 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687816e-7d38-42d9-a1b4-140cd15914e6 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:27.465189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4687816e-7d38-42d9-a1b4-140cd15914e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:27.477960 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5b70a4d5-a2a5-4651-803a-d8686348d914 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba0744a7da2c41c194bbbe8774bbaeb7'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.479883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687816e-7d38-42d9-a1b4-140cd15914e6 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:27.485285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643e7b7-c1e8-4ab0-b664-177dd6d5e56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:27.486242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643e7b7-c1e8-4ab0-b664-177dd6d5e56b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:27.486500 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643e7b7-c1e8-4ab0-b664-177dd6d5e56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:27.486892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643e7b7-c1e8-4ab0-b664-177dd6d5e56b admin admin] PATH_INFO: `/v3/projects/b10e877cd8eb4f2f948db4ac229f85d5` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:27.488671 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643e7b7-c1e8-4ab0-b664-177dd6d5e56b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10e877cd8eb4f2f948db4ac229f85d5)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:27.490418 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643e7b7-c1e8-4ab0-b664-177dd6d5e56b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:27.494374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5b70a4d5-a2a5-4651-803a-d8686348d914 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba0744a7da2c41c194bbbe8774bbaeb7'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.494674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5b70a4d5-a2a5-4651-803a-d8686348d914 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba0744a7da2c41c194bbbe8774bbaeb7'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.495365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5b70a4d5-a2a5-4651-803a-d8686348d914 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba0744a7da2c41c194bbbe8774bbaeb7'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.500830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:27.522487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1139/2273] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:27 2022] DELETE /identity/v3/users/ba0744a7da2c41c194bbbe8774bbaeb7 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:45:27.562841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e643e7b7-c1e8-4ab0-b664-177dd6d5e56b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10e877cd8eb4f2f948db4ac229f85d5', 'request_id': 'req-e643e7b7-c1e8-4ab0-b664-177dd6d5e56b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.572143 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e643e7b7-c1e8-4ab0-b664-177dd6d5e56b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10e877cd8eb4f2f948db4ac229f85d5', 'request_id': 'req-e643e7b7-c1e8-4ab0-b664-177dd6d5e56b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.579760 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e643e7b7-c1e8-4ab0-b664-177dd6d5e56b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10e877cd8eb4f2f948db4ac229f85d5', 'request_id': 'req-e643e7b7-c1e8-4ab0-b664-177dd6d5e56b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.597038 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e643e7b7-c1e8-4ab0-b664-177dd6d5e56b admin admin] The token cache is being invalidate because project b10e877cd8eb4f2f948db4ac229f85d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:27.597623 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e643e7b7-c1e8-4ab0-b664-177dd6d5e56b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.598437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-e643e7b7-c1e8-4ab0-b664-177dd6d5e56b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.600358 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1135/2274] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:27 2022] DELETE /identity/v3/projects/b10e877cd8eb4f2f948db4ac229f85d5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:27.607417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7c4b7-f414-4e23-9cf8-597fdac1d5a9 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:27.607944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7c4b7-f414-4e23-9cf8-597fdac1d5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:27.608364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7c4b7-f414-4e23-9cf8-597fdac1d5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:27.632817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-84b7c4b7-f414-4e23-9cf8-597fdac1d5a9 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:27.632817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-84b7c4b7-f414-4e23-9cf8-597fdac1d5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:27.728336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1140/2275] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:27.736149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc2621-c9c9-4548-b166-f11f55b7f15a None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:27.736461 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc2621-c9c9-4548-b166-f11f55b7f15a None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:27.736822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc2621-c9c9-4548-b166-f11f55b7f15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:27.756827 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-57fc2621-c9c9-4548-b166-f11f55b7f15a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:27.758075 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-57fc2621-c9c9-4548-b166-f11f55b7f15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:27.768191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1136/2276] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:27.773024 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7c4b7-f414-4e23-9cf8-597fdac1d5a9 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:27.785301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7c4b7-f414-4e23-9cf8-597fdac1d5a9 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:27.791414 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abac24-bd2d-4e6d-b82f-a5e05c334e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:27.792061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abac24-bd2d-4e6d-b82f-a5e05c334e3f admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:27.792346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abac24-bd2d-4e6d-b82f-a5e05c334e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:27.792624 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abac24-bd2d-4e6d-b82f-a5e05c334e3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:27.793530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abac24-bd2d-4e6d-b82f-a5e05c334e3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:45:27.793975 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abac24-bd2d-4e6d-b82f-a5e05c334e3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:27.796357 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abac24-bd2d-4e6d-b82f-a5e05c334e3f admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:27.813248 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1141/2277] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:45:27.814379 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc2621-c9c9-4548-b166-f11f55b7f15a None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:27.824894 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc2621-c9c9-4548-b166-f11f55b7f15a None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:27.837643 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90bc5c5-4f6d-491a-8df6-bd408b75fde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:27.838384 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bc5c5-4f6d-491a-8df6-bd408b75fde3 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:27.838687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bc5c5-4f6d-491a-8df6-bd408b75fde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:27.839007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90bc5c5-4f6d-491a-8df6-bd408b75fde3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:27.840516 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90bc5c5-4f6d-491a-8df6-bd408b75fde3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:27.842594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90bc5c5-4f6d-491a-8df6-bd408b75fde3 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:27.847111 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abac24-bd2d-4e6d-b82f-a5e05c334e3f admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:27.849608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e7abac24-bd2d-4e6d-b82f-a5e05c334e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:27.869786 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abac24-bd2d-4e6d-b82f-a5e05c334e3f admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:27.876745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d0a40-52e9-4fc4-9182-b26bf0811842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:27.877963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d0a40-52e9-4fc4-9182-b26bf0811842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:27.878317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d0a40-52e9-4fc4-9182-b26bf0811842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:27.878763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d0a40-52e9-4fc4-9182-b26bf0811842 admin admin] PATH_INFO: `/v3/projects/b123cefb46304e5a9f09eee3770a1978` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:27.879554 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1137/2278] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:45:27 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:45:27.883783 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d0a40-52e9-4fc4-9182-b26bf0811842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b123cefb46304e5a9f09eee3770a1978)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:27.884018 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90bc5c5-4f6d-491a-8df6-bd408b75fde3 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:27.886216 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d0a40-52e9-4fc4-9182-b26bf0811842 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:27.936349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0b0d0a40-52e9-4fc4-9182-b26bf0811842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b123cefb46304e5a9f09eee3770a1978', 'request_id': 'req-0b0d0a40-52e9-4fc4-9182-b26bf0811842'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.942618 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0b0d0a40-52e9-4fc4-9182-b26bf0811842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b123cefb46304e5a9f09eee3770a1978', 'request_id': 'req-0b0d0a40-52e9-4fc4-9182-b26bf0811842'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.950852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90bc5c5-4f6d-491a-8df6-bd408b75fde3 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:27.951858 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0b0d0a40-52e9-4fc4-9182-b26bf0811842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b123cefb46304e5a9f09eee3770a1978', 'request_id': 'req-0b0d0a40-52e9-4fc4-9182-b26bf0811842'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.964105 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0b0d0a40-52e9-4fc4-9182-b26bf0811842 admin admin] The token cache is being invalidate because project b123cefb46304e5a9f09eee3770a1978 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:27.964397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0b0d0a40-52e9-4fc4-9182-b26bf0811842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.965124 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-0b0d0a40-52e9-4fc4-9182-b26bf0811842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:27.967167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1142/2279] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:27 2022] DELETE /identity/v3/projects/b123cefb46304e5a9f09eee3770a1978 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:27.972406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d0a40-52e9-4fc4-9182-b26bf0811842 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:27.974073 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-0b0d0a40-52e9-4fc4-9182-b26bf0811842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:27.997927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:28.012336 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a835af-6f12-4aa8-b04f-32347cd4ca24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:28.013053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a835af-6f12-4aa8-b04f-32347cd4ca24 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:28.013406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a835af-6f12-4aa8-b04f-32347cd4ca24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:28.013663 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a835af-6f12-4aa8-b04f-32347cd4ca24 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:28.014502 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a835af-6f12-4aa8-b04f-32347cd4ca24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:28.016476 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a835af-6f12-4aa8-b04f-32347cd4ca24 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:28.042340 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1138/2280] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:45:27 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:45:28.043458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a835af-6f12-4aa8-b04f-32347cd4ca24 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:28.045173 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a0a835af-6f12-4aa8-b04f-32347cd4ca24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:28.055814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d0a40-52e9-4fc4-9182-b26bf0811842 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:28.057287 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a835af-6f12-4aa8-b04f-32347cd4ca24 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:28.060825 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a4662-85c7-436a-9e1e-f87c386809bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:28.061730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a4662-85c7-436a-9e1e-f87c386809bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:28.062014 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a4662-85c7-436a-9e1e-f87c386809bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:28.062285 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a4662-85c7-436a-9e1e-f87c386809bd admin admin] PATH_INFO: `/v3/users/416d44c244a44e4bae52dc3c1bc7e285` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:28.063532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a4662-85c7-436a-9e1e-f87c386809bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=416d44c244a44e4bae52dc3c1bc7e285)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:28.063815 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99f436-5217-4d54-9d14-105704c9b0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:28.064605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99f436-5217-4d54-9d14-105704c9b0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:28.064965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99f436-5217-4d54-9d14-105704c9b0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:28.065243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99f436-5217-4d54-9d14-105704c9b0a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:28.065588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a4662-85c7-436a-9e1e-f87c386809bd admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:28.065983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99f436-5217-4d54-9d14-105704c9b0a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:28.067628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99f436-5217-4d54-9d14-105704c9b0a9 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:28.073619 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:28.082374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1139/2281] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:45:28.083409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99f436-5217-4d54-9d14-105704c9b0a9 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:28.095852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99f436-5217-4d54-9d14-105704c9b0a9 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:28.100999 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23a1df-b008-4f24-9302-eefce06dbb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:28.101825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23a1df-b008-4f24-9302-eefce06dbb2f admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:28.102112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23a1df-b008-4f24-9302-eefce06dbb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:28.102388 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23a1df-b008-4f24-9302-eefce06dbb2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:28.103350 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23a1df-b008-4f24-9302-eefce06dbb2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:28.104996 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23a1df-b008-4f24-9302-eefce06dbb2f admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:28.118263 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1140/2282] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:28.122228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23a1df-b008-4f24-9302-eefce06dbb2f admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:28.122391 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-195a4662-85c7-436a-9e1e-f87c386809bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '416d44c244a44e4bae52dc3c1bc7e285'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:28.133224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23a1df-b008-4f24-9302-eefce06dbb2f admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:28.137539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-195a4662-85c7-436a-9e1e-f87c386809bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '416d44c244a44e4bae52dc3c1bc7e285'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:28.139932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce9815-a45c-4127-afbe-1a37e9638488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:28.141245 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce9815-a45c-4127-afbe-1a37e9638488 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:28.141513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce9815-a45c-4127-afbe-1a37e9638488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:28.141792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce9815-a45c-4127-afbe-1a37e9638488 admin admin] PATH_INFO: `/v3/projects/3f2cc047ebb946e78ea556dfb0ca3468/users/45198a704d7c4ff79c21d10cf2057074/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:28.147839 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-195a4662-85c7-436a-9e1e-f87c386809bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '416d44c244a44e4bae52dc3c1bc7e285'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:28.149079 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-195a4662-85c7-436a-9e1e-f87c386809bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '416d44c244a44e4bae52dc3c1bc7e285'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:28.150233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-195a4662-85c7-436a-9e1e-f87c386809bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '416d44c244a44e4bae52dc3c1bc7e285'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:28.161849 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce9815-a45c-4127-afbe-1a37e9638488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f2cc047ebb946e78ea556dfb0ca3468, user_id=45198a704d7c4ff79c21d10cf2057074, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:28.164784 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce9815-a45c-4127-afbe-1a37e9638488 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:28.168558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1143/2283] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:27 2022] DELETE /identity/v3/users/416d44c244a44e4bae52dc3c1bc7e285 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:28.181890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1141/2284] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:28 2022] PUT /identity/v3/projects/3f2cc047ebb946e78ea556dfb0ca3468/users/45198a704d7c4ff79c21d10cf2057074/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:45:28.192652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed065eca-19de-44c7-a557-a041f3a5f61b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:28.193035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed065eca-19de-44c7-a557-a041f3a5f61b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:28.193331 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed065eca-19de-44c7-a557-a041f3a5f61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:28.223444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-ed065eca-19de-44c7-a557-a041f3a5f61b None None] MFA Rules not processed for user `45198a704d7c4ff79c21d10cf2057074`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:28.224818 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ed065eca-19de-44c7-a557-a041f3a5f61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:28.313150 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1144/2285] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:28.324589 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce9815-a45c-4127-afbe-1a37e9638488 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:28.326403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-90ce9815-a45c-4127-afbe-1a37e9638488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:28.387345 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce9815-a45c-4127-afbe-1a37e9638488 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:28.391537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea06b48-d32b-4f52-a599-ce60f429c424 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:28.392208 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea06b48-d32b-4f52-a599-ce60f429c424 service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:28.392499 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea06b48-d32b-4f52-a599-ce60f429c424 service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:28.392801 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea06b48-d32b-4f52-a599-ce60f429c424 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:28.401920 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea06b48-d32b-4f52-a599-ce60f429c424 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:28.403424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea06b48-d32b-4f52-a599-ce60f429c424 service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:28.413085 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1142/2286] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:28.459279 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed065eca-19de-44c7-a557-a041f3a5f61b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:28.460808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ed065eca-19de-44c7-a557-a041f3a5f61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:28.546751 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed065eca-19de-44c7-a557-a041f3a5f61b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:28.552892 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad150b-daf6-4afd-bf85-bd174e2b7c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:28.553925 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad150b-daf6-4afd-bf85-bd174e2b7c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:28.554210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad150b-daf6-4afd-bf85-bd174e2b7c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:28.554486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad150b-daf6-4afd-bf85-bd174e2b7c96 admin admin] PATH_INFO: `/v3/projects/c8633ff85f9c4510a949645436a0210e` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:28.556675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad150b-daf6-4afd-bf85-bd174e2b7c96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8633ff85f9c4510a949645436a0210e)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:28.558825 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad150b-daf6-4afd-bf85-bd174e2b7c96 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:28.579348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:28.614110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c0ad150b-daf6-4afd-bf85-bd174e2b7c96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8633ff85f9c4510a949645436a0210e', 'request_id': 'req-c0ad150b-daf6-4afd-bf85-bd174e2b7c96'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:28.619780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c0ad150b-daf6-4afd-bf85-bd174e2b7c96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8633ff85f9c4510a949645436a0210e', 'request_id': 'req-c0ad150b-daf6-4afd-bf85-bd174e2b7c96'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:28.628097 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c0ad150b-daf6-4afd-bf85-bd174e2b7c96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8633ff85f9c4510a949645436a0210e', 'request_id': 'req-c0ad150b-daf6-4afd-bf85-bd174e2b7c96'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:28.640539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c0ad150b-daf6-4afd-bf85-bd174e2b7c96 admin admin] The token cache is being invalidate because project c8633ff85f9c4510a949645436a0210e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:28.640854 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c0ad150b-daf6-4afd-bf85-bd174e2b7c96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:28.641480 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-c0ad150b-daf6-4afd-bf85-bd174e2b7c96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:28.643416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1145/2287] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:28 2022] DELETE /identity/v3/projects/c8633ff85f9c4510a949645436a0210e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:28.653071 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cb0cc-555a-47d2-99b6-e2879db996b1 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:28.653476 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cb0cc-555a-47d2-99b6-e2879db996b1 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:28.653683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cb0cc-555a-47d2-99b6-e2879db996b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:28.677273 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-164cb0cc-555a-47d2-99b6-e2879db996b1 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:28.678674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-164cb0cc-555a-47d2-99b6-e2879db996b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:28.763369 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1143/2288] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:28.767049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f0e37-1ff5-49d5-ad55-339775e18938 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:28.767500 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f0e37-1ff5-49d5-ad55-339775e18938 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:28.768289 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f0e37-1ff5-49d5-ad55-339775e18938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:28.806286 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-9d6f0e37-1ff5-49d5-ad55-339775e18938 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:28.807942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-9d6f0e37-1ff5-49d5-ad55-339775e18938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:28.821039 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1146/2289] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:28.822288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cb0cc-555a-47d2-99b6-e2879db996b1 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:28.833232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cb0cc-555a-47d2-99b6-e2879db996b1 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:28.841706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f016ea-0678-4447-9874-a14643ebfc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:28.842485 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f016ea-0678-4447-9874-a14643ebfc5a admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:28.842775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f016ea-0678-4447-9874-a14643ebfc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:28.843133 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f016ea-0678-4447-9874-a14643ebfc5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:28.844497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f016ea-0678-4447-9874-a14643ebfc5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:45:28.844826 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f016ea-0678-4447-9874-a14643ebfc5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:28.846965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f016ea-0678-4447-9874-a14643ebfc5a admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:28.863087 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1144/2290] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:45:28.867233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f0e37-1ff5-49d5-ad55-339775e18938 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:28.880350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f0e37-1ff5-49d5-ad55-339775e18938 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:28.886814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791555e-984e-4506-8e74-1c6d8636c8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:28.887724 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791555e-984e-4506-8e74-1c6d8636c8a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:28.888075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791555e-984e-4506-8e74-1c6d8636c8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:28.888629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791555e-984e-4506-8e74-1c6d8636c8a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:28.889748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791555e-984e-4506-8e74-1c6d8636c8a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:28.892837 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791555e-984e-4506-8e74-1c6d8636c8a1 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:28.925984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1147/2291] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:45:28 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:28.927705 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f016ea-0678-4447-9874-a14643ebfc5a admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:28.938546 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f016ea-0678-4447-9874-a14643ebfc5a admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:28.961803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:29.010593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad99c2-ad41-4218-975a-e9dda842de39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:29.011323 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad99c2-ad41-4218-975a-e9dda842de39 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:29.011613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad99c2-ad41-4218-975a-e9dda842de39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:29.011941 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad99c2-ad41-4218-975a-e9dda842de39 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:29.012659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad99c2-ad41-4218-975a-e9dda842de39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:29.015290 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad99c2-ad41-4218-975a-e9dda842de39 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:29.038783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1145/2292] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:45:28 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:45:29.042950 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791555e-984e-4506-8e74-1c6d8636c8a1 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:29.053723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791555e-984e-4506-8e74-1c6d8636c8a1 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:29.059557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e994609-0f90-40bd-ae17-74154c3100d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:29.060231 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e994609-0f90-40bd-ae17-74154c3100d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:29.060519 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e994609-0f90-40bd-ae17-74154c3100d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:29.060799 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e994609-0f90-40bd-ae17-74154c3100d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:29.061544 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e994609-0f90-40bd-ae17-74154c3100d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:29.063241 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e994609-0f90-40bd-ae17-74154c3100d3 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:29.074509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1148/2293] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:29.078828 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad99c2-ad41-4218-975a-e9dda842de39 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:29.088975 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad99c2-ad41-4218-975a-e9dda842de39 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:29.094282 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da322acb-ec6a-424c-a660-839bf76d79ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:29.094943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da322acb-ec6a-424c-a660-839bf76d79ff admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:29.095259 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da322acb-ec6a-424c-a660-839bf76d79ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:29.095536 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da322acb-ec6a-424c-a660-839bf76d79ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:29.096293 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da322acb-ec6a-424c-a660-839bf76d79ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:29.097924 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da322acb-ec6a-424c-a660-839bf76d79ff admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:29.108057 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1146/2294] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:29.112276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e994609-0f90-40bd-ae17-74154c3100d3 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:29.123535 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e994609-0f90-40bd-ae17-74154c3100d3 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:29.130637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229cf544-c23e-4347-9747-1c56b5add7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:29.131579 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cf544-c23e-4347-9747-1c56b5add7b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:29.131863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cf544-c23e-4347-9747-1c56b5add7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:29.132135 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cf544-c23e-4347-9747-1c56b5add7b9 admin admin] PATH_INFO: `/v3/projects/1e7f63a3eff14f8ebbebc179c641038d/users/e2fc546cbc874afcb34339d3efd37ace/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:29.150242 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229cf544-c23e-4347-9747-1c56b5add7b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e7f63a3eff14f8ebbebc179c641038d, user_id=e2fc546cbc874afcb34339d3efd37ace, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:29.153192 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229cf544-c23e-4347-9747-1c56b5add7b9 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:29.170078 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1149/2295] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:29 2022] PUT /identity/v3/projects/1e7f63a3eff14f8ebbebc179c641038d/users/e2fc546cbc874afcb34339d3efd37ace/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:45:29.180488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcf9b8-36c2-4441-9030-a9053fcec076 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:29.180981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcf9b8-36c2-4441-9030-a9053fcec076 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:29.181243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcf9b8-36c2-4441-9030-a9053fcec076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:29.211948 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-b5fcf9b8-36c2-4441-9030-a9053fcec076 None None] MFA Rules not processed for user `e2fc546cbc874afcb34339d3efd37ace`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:29.213318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b5fcf9b8-36c2-4441-9030-a9053fcec076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:29.287190 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1147/2296] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:29.300999 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229cf544-c23e-4347-9747-1c56b5add7b9 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:29.305044 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-229cf544-c23e-4347-9747-1c56b5add7b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:29.390705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229cf544-c23e-4347-9747-1c56b5add7b9 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:29.394596 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ace11a-d59e-4d80-a366-0b7c6361cc4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:29.395287 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ace11a-d59e-4d80-a366-0b7c6361cc4c service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:29.395569 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ace11a-d59e-4d80-a366-0b7c6361cc4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:29.395844 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ace11a-d59e-4d80-a366-0b7c6361cc4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:29.405363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ace11a-d59e-4d80-a366-0b7c6361cc4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:29.406952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ace11a-d59e-4d80-a366-0b7c6361cc4c service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:29.419119 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1150/2297] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:29.449473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fcf9b8-36c2-4441-9030-a9053fcec076 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:29.537150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fcf9b8-36c2-4441-9030-a9053fcec076 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:29.544408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e30f91-7404-4c62-9137-6de549b30545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:29.545552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e30f91-7404-4c62-9137-6de549b30545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:29.547313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e30f91-7404-4c62-9137-6de549b30545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:29.547598 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e30f91-7404-4c62-9137-6de549b30545 admin admin] PATH_INFO: `/v3/users/e2fc546cbc874afcb34339d3efd37ace` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:29.549205 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e30f91-7404-4c62-9137-6de549b30545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2fc546cbc874afcb34339d3efd37ace)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:29.551295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e30f91-7404-4c62-9137-6de549b30545 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:29.610985 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:29.625648 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-75e30f91-7404-4c62-9137-6de549b30545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2fc546cbc874afcb34339d3efd37ace'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:29.638331 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-75e30f91-7404-4c62-9137-6de549b30545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2fc546cbc874afcb34339d3efd37ace'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:29.646137 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-75e30f91-7404-4c62-9137-6de549b30545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2fc546cbc874afcb34339d3efd37ace'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:29.651597 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-75e30f91-7404-4c62-9137-6de549b30545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2fc546cbc874afcb34339d3efd37ace'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:29.653056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-75e30f91-7404-4c62-9137-6de549b30545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2fc546cbc874afcb34339d3efd37ace'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:29.671509 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1148/2298] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:29 2022] DELETE /identity/v3/users/e2fc546cbc874afcb34339d3efd37ace => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:29.676234 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ac18b-0b6a-4ff1-9200-1ed5fca4d21d None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:29.676529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ac18b-0b6a-4ff1-9200-1ed5fca4d21d None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:29.676809 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ac18b-0b6a-4ff1-9200-1ed5fca4d21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:29.698171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-789ac18b-0b6a-4ff1-9200-1ed5fca4d21d None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:29.699422 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-789ac18b-0b6a-4ff1-9200-1ed5fca4d21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:29.752299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:29.774407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1151/2299] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:29.786008 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e30f91-7404-4c62-9137-6de549b30545 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:29.787573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-75e30f91-7404-4c62-9137-6de549b30545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:29.857578 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e30f91-7404-4c62-9137-6de549b30545 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:29.864137 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed098280-6931-4f74-b56f-0daa000b4602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:29.864802 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed098280-6931-4f74-b56f-0daa000b4602 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:29.865157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed098280-6931-4f74-b56f-0daa000b4602 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:29.865431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed098280-6931-4f74-b56f-0daa000b4602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:29.876348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed098280-6931-4f74-b56f-0daa000b4602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:29.877966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed098280-6931-4f74-b56f-0daa000b4602 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:29.889151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1149/2300] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:45:30.212662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ac18b-0b6a-4ff1-9200-1ed5fca4d21d None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:30.216614 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-789ac18b-0b6a-4ff1-9200-1ed5fca4d21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:30.235397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ac18b-0b6a-4ff1-9200-1ed5fca4d21d None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:30.245350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca816b7-774f-434d-b52e-0a5d98122fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:30.246753 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca816b7-774f-434d-b52e-0a5d98122fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:30.247250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca816b7-774f-434d-b52e-0a5d98122fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:30.247616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca816b7-774f-434d-b52e-0a5d98122fc2 admin admin] PATH_INFO: `/v3/projects/1e7f63a3eff14f8ebbebc179c641038d` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:30.249149 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca816b7-774f-434d-b52e-0a5d98122fc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e7f63a3eff14f8ebbebc179c641038d)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:30.251524 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca816b7-774f-434d-b52e-0a5d98122fc2 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:30.309031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-cca816b7-774f-434d-b52e-0a5d98122fc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e7f63a3eff14f8ebbebc179c641038d', 'request_id': 'req-cca816b7-774f-434d-b52e-0a5d98122fc2'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:30.316618 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-cca816b7-774f-434d-b52e-0a5d98122fc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e7f63a3eff14f8ebbebc179c641038d', 'request_id': 'req-cca816b7-774f-434d-b52e-0a5d98122fc2'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:30.325376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-cca816b7-774f-434d-b52e-0a5d98122fc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e7f63a3eff14f8ebbebc179c641038d', 'request_id': 'req-cca816b7-774f-434d-b52e-0a5d98122fc2'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:30.337427 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed098280-6931-4f74-b56f-0daa000b4602 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:30.338978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ed098280-6931-4f74-b56f-0daa000b4602 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:30.341092 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-cca816b7-774f-434d-b52e-0a5d98122fc2 admin admin] The token cache is being invalidate because project 1e7f63a3eff14f8ebbebc179c641038d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:30.341475 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-cca816b7-774f-434d-b52e-0a5d98122fc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:30.342220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-cca816b7-774f-434d-b52e-0a5d98122fc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:30.353650 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed098280-6931-4f74-b56f-0daa000b4602 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:30.357190 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1152/2301] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:30 2022] DELETE /identity/v3/projects/1e7f63a3eff14f8ebbebc179c641038d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:30.430899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009165bf-fa8a-445b-b85a-f260133dec1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:30.432079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009165bf-fa8a-445b-b85a-f260133dec1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:30.432432 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009165bf-fa8a-445b-b85a-f260133dec1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:30.432929 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009165bf-fa8a-445b-b85a-f260133dec1f admin admin] PATH_INFO: `/v3/users/b5deb44456514ca98a43d5d49521d65d` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:30.434520 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009165bf-fa8a-445b-b85a-f260133dec1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5deb44456514ca98a43d5d49521d65d)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:30.436587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009165bf-fa8a-445b-b85a-f260133dec1f admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:30.515505 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-009165bf-fa8a-445b-b85a-f260133dec1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5deb44456514ca98a43d5d49521d65d'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:30.529504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:30.551773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-009165bf-fa8a-445b-b85a-f260133dec1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5deb44456514ca98a43d5d49521d65d'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:30.562456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-009165bf-fa8a-445b-b85a-f260133dec1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5deb44456514ca98a43d5d49521d65d'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:30.563390 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-009165bf-fa8a-445b-b85a-f260133dec1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5deb44456514ca98a43d5d49521d65d'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:30.564000 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-009165bf-fa8a-445b-b85a-f260133dec1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5deb44456514ca98a43d5d49521d65d'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:30.584763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1150/2302] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:30 2022] DELETE /identity/v3/users/b5deb44456514ca98a43d5d49521d65d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:30.590308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5127236-2429-474c-a501-090e20b2f7a8 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:30.590925 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5127236-2429-474c-a501-090e20b2f7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:30.591306 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5127236-2429-474c-a501-090e20b2f7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:30.613960 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-c5127236-2429-474c-a501-090e20b2f7a8 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:30.615978 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c5127236-2429-474c-a501-090e20b2f7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:30.687877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1153/2303] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:30.699605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009165bf-fa8a-445b-b85a-f260133dec1f admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:30.701133 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-009165bf-fa8a-445b-b85a-f260133dec1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:30.759895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009165bf-fa8a-445b-b85a-f260133dec1f admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:30.763682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a86550-a8f1-4e7b-91af-d3f529808784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:30.764443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a86550-a8f1-4e7b-91af-d3f529808784 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:30.764761 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a86550-a8f1-4e7b-91af-d3f529808784 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:30.765076 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a86550-a8f1-4e7b-91af-d3f529808784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:30.775802 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a86550-a8f1-4e7b-91af-d3f529808784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:30.777268 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a86550-a8f1-4e7b-91af-d3f529808784 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:30.787531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1151/2304] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:31.087135 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5127236-2429-474c-a501-090e20b2f7a8 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:31.088704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c5127236-2429-474c-a501-090e20b2f7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:31.102296 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5127236-2429-474c-a501-090e20b2f7a8 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:31.107199 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:31.108240 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:31.108682 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:31.109090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6 admin admin] PATH_INFO: `/v3/projects/452eab20124f4fa1aadb94173c8929f9` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:31.110284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=452eab20124f4fa1aadb94173c8929f9)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:31.112651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:31.163951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '452eab20124f4fa1aadb94173c8929f9', 'request_id': 'req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:31.171439 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '452eab20124f4fa1aadb94173c8929f9', 'request_id': 'req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:31.175339 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:31.181144 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '452eab20124f4fa1aadb94173c8929f9', 'request_id': 'req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:31.196185 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6 admin admin] The token cache is being invalidate because project 452eab20124f4fa1aadb94173c8929f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:31.196476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:31.197123 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:31.198694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1154/2305] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:31 2022] DELETE /identity/v3/projects/452eab20124f4fa1aadb94173c8929f9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:31.202720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a86550-a8f1-4e7b-91af-d3f529808784 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:31.215596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a86550-a8f1-4e7b-91af-d3f529808784 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:31.281362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed907fc-f4d8-4cec-81b1-28c4c03bfe2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:31.282259 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed907fc-f4d8-4cec-81b1-28c4c03bfe2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:31.282538 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed907fc-f4d8-4cec-81b1-28c4c03bfe2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:31.282870 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed907fc-f4d8-4cec-81b1-28c4c03bfe2f admin admin] PATH_INFO: `/v3/users/c7cb83e0b00343b59593d38ec8b340dc` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:31.284067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed907fc-f4d8-4cec-81b1-28c4c03bfe2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7cb83e0b00343b59593d38ec8b340dc)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:31.286524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed907fc-f4d8-4cec-81b1-28c4c03bfe2f admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:31.340078 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-6ed907fc-f4d8-4cec-81b1-28c4c03bfe2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7cb83e0b00343b59593d38ec8b340dc'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:31.356221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-6ed907fc-f4d8-4cec-81b1-28c4c03bfe2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7cb83e0b00343b59593d38ec8b340dc'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:31.363632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-6ed907fc-f4d8-4cec-81b1-28c4c03bfe2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7cb83e0b00343b59593d38ec8b340dc'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:31.364356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-6ed907fc-f4d8-4cec-81b1-28c4c03bfe2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7cb83e0b00343b59593d38ec8b340dc'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:31.364967 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-6ed907fc-f4d8-4cec-81b1-28c4c03bfe2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7cb83e0b00343b59593d38ec8b340dc'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:31.367740 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:31.383845 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1152/2306] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:31 2022] DELETE /identity/v3/users/c7cb83e0b00343b59593d38ec8b340dc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:31.696065 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:31.697599 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:31.711542 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07056a-8a02-4dc5-a9c8-e06849f3f4a6 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:31.775736 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e35def-c756-43ac-bae9-bcb0d72f7e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:31.776634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e35def-c756-43ac-bae9-bcb0d72f7e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:31.776920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e35def-c756-43ac-bae9-bcb0d72f7e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:31.777194 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e35def-c756-43ac-bae9-bcb0d72f7e44 admin admin] PATH_INFO: `/v3/projects/a413be817ae147f9815de51a8bb7352e` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:31.778545 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e35def-c756-43ac-bae9-bcb0d72f7e44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a413be817ae147f9815de51a8bb7352e)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:31.780561 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e35def-c756-43ac-bae9-bcb0d72f7e44 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:31.833271 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-54e35def-c756-43ac-bae9-bcb0d72f7e44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a413be817ae147f9815de51a8bb7352e', 'request_id': 'req-54e35def-c756-43ac-bae9-bcb0d72f7e44'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:31.838743 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-54e35def-c756-43ac-bae9-bcb0d72f7e44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a413be817ae147f9815de51a8bb7352e', 'request_id': 'req-54e35def-c756-43ac-bae9-bcb0d72f7e44'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:31.847879 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-54e35def-c756-43ac-bae9-bcb0d72f7e44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a413be817ae147f9815de51a8bb7352e', 'request_id': 'req-54e35def-c756-43ac-bae9-bcb0d72f7e44'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:31.864651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-54e35def-c756-43ac-bae9-bcb0d72f7e44 admin admin] The token cache is being invalidate because project a413be817ae147f9815de51a8bb7352e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:31.864838 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-54e35def-c756-43ac-bae9-bcb0d72f7e44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:31.865711 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-54e35def-c756-43ac-bae9-bcb0d72f7e44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:31.868111 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1155/2307] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:31 2022] DELETE /identity/v3/projects/a413be817ae147f9815de51a8bb7352e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:31.873905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc278e26-1295-4662-a9c2-77115480b07b None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:31.874222 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc278e26-1295-4662-a9c2-77115480b07b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:31.874564 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc278e26-1295-4662-a9c2-77115480b07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:31.894599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-cc278e26-1295-4662-a9c2-77115480b07b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:31.895833 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-cc278e26-1295-4662-a9c2-77115480b07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:31.967027 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1153/2308] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:31.976318 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468812a2-9a02-44ad-8a84-1c7ffb8bcd5a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:31.976658 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468812a2-9a02-44ad-8a84-1c7ffb8bcd5a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:31.976976 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468812a2-9a02-44ad-8a84-1c7ffb8bcd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:31.999831 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-468812a2-9a02-44ad-8a84-1c7ffb8bcd5a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:32.001046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-468812a2-9a02-44ad-8a84-1c7ffb8bcd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:32.010588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1156/2309] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:32.014871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc278e26-1295-4662-a9c2-77115480b07b None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:32.025407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc278e26-1295-4662-a9c2-77115480b07b None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:32.031314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bdb57b-1467-4a4c-811f-d3b89501b7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:32.031975 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdb57b-1467-4a4c-811f-d3b89501b7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:32.032255 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdb57b-1467-4a4c-811f-d3b89501b7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:32.032529 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdb57b-1467-4a4c-811f-d3b89501b7f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:32.033263 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bdb57b-1467-4a4c-811f-d3b89501b7f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:45:32.033571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bdb57b-1467-4a4c-811f-d3b89501b7f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:32.035618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bdb57b-1467-4a4c-811f-d3b89501b7f4 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:32.047971 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1154/2310] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:45:32.052598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468812a2-9a02-44ad-8a84-1c7ffb8bcd5a None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:32.062139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468812a2-9a02-44ad-8a84-1c7ffb8bcd5a None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:32.068015 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a08fa3-02e7-4cdb-9297-6c513621c665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:32.068677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a08fa3-02e7-4cdb-9297-6c513621c665 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:32.068970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a08fa3-02e7-4cdb-9297-6c513621c665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:32.069233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a08fa3-02e7-4cdb-9297-6c513621c665 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:32.069933 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a08fa3-02e7-4cdb-9297-6c513621c665 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:32.071906 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a08fa3-02e7-4cdb-9297-6c513621c665 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:32.102841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1157/2311] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:45:32 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:32.104057 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bdb57b-1467-4a4c-811f-d3b89501b7f4 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:32.114321 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bdb57b-1467-4a4c-811f-d3b89501b7f4 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:32.172614 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c29da0-7458-488d-a652-603c7ea22112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:32.173296 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c29da0-7458-488d-a652-603c7ea22112 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:32.173594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c29da0-7458-488d-a652-603c7ea22112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:32.173861 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c29da0-7458-488d-a652-603c7ea22112 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:32.174637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c29da0-7458-488d-a652-603c7ea22112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:32.176680 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c29da0-7458-488d-a652-603c7ea22112 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:32.198156 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1155/2312] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:45:32 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:45:32.204358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a08fa3-02e7-4cdb-9297-6c513621c665 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:32.215802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a08fa3-02e7-4cdb-9297-6c513621c665 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:32.221313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d626647-f5ee-42b8-9301-3e59466e4fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:32.221999 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d626647-f5ee-42b8-9301-3e59466e4fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:32.222394 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d626647-f5ee-42b8-9301-3e59466e4fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:32.222678 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d626647-f5ee-42b8-9301-3e59466e4fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:32.223466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d626647-f5ee-42b8-9301-3e59466e4fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:32.225125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d626647-f5ee-42b8-9301-3e59466e4fc7 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:32.238021 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1158/2313] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:32.243344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c29da0-7458-488d-a652-603c7ea22112 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:32.253812 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c29da0-7458-488d-a652-603c7ea22112 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:32.261443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53762b8-4ec5-40fc-9ca3-c9edf83adff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:32.262409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53762b8-4ec5-40fc-9ca3-c9edf83adff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:32.262691 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53762b8-4ec5-40fc-9ca3-c9edf83adff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:32.263034 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53762b8-4ec5-40fc-9ca3-c9edf83adff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:32.264147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53762b8-4ec5-40fc-9ca3-c9edf83adff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:32.266656 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53762b8-4ec5-40fc-9ca3-c9edf83adff4 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:32.277434 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1156/2314] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:32.281617 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d626647-f5ee-42b8-9301-3e59466e4fc7 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:32.292761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d626647-f5ee-42b8-9301-3e59466e4fc7 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:32.299684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc9b6d-4281-4c1c-9ac6-a03f3121a1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:32.300578 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc9b6d-4281-4c1c-9ac6-a03f3121a1cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:32.300820 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc9b6d-4281-4c1c-9ac6-a03f3121a1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:32.301122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc9b6d-4281-4c1c-9ac6-a03f3121a1cd admin admin] PATH_INFO: `/v3/projects/a553b56b6b834dcbaa8540c8c8070cee/users/8a7fc3fd922440219dabcac851223363/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:32.317371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc9b6d-4281-4c1c-9ac6-a03f3121a1cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a553b56b6b834dcbaa8540c8c8070cee, user_id=8a7fc3fd922440219dabcac851223363, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:32.320110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc9b6d-4281-4c1c-9ac6-a03f3121a1cd admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:32.342322 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1159/2315] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:32 2022] PUT /identity/v3/projects/a553b56b6b834dcbaa8540c8c8070cee/users/8a7fc3fd922440219dabcac851223363/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:45:32.346174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6ea0e-9859-406b-98c6-ee78d18b7081 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:32.346676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6ea0e-9859-406b-98c6-ee78d18b7081 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:32.346676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6ea0e-9859-406b-98c6-ee78d18b7081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:32.374200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-88c6ea0e-9859-406b-98c6-ee78d18b7081 None None] MFA Rules not processed for user `8a7fc3fd922440219dabcac851223363`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:32.375360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-88c6ea0e-9859-406b-98c6-ee78d18b7081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:32.421609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:32.441606 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1157/2316] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:32.446288 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc9b6d-4281-4c1c-9ac6-a03f3121a1cd admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:32.470292 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:32.522188 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc9b6d-4281-4c1c-9ac6-a03f3121a1cd admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:32.533206 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a03db5-4cb7-4c60-9793-80516c601f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:32.533206 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a03db5-4cb7-4c60-9793-80516c601f15 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:32.533206 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a03db5-4cb7-4c60-9793-80516c601f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:32.533206 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a03db5-4cb7-4c60-9793-80516c601f15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:32.533652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a03db5-4cb7-4c60-9793-80516c601f15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:32.533652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a03db5-4cb7-4c60-9793-80516c601f15 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:32.557233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1160/2317] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:45:32 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:32.561558 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6ea0e-9859-406b-98c6-ee78d18b7081 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:32.572456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6ea0e-9859-406b-98c6-ee78d18b7081 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:32.632801 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcca849-ddf5-4b45-8af6-12271d7ece4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:32.633484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcca849-ddf5-4b45-8af6-12271d7ece4a admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:32.634054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcca849-ddf5-4b45-8af6-12271d7ece4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:32.634054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcca849-ddf5-4b45-8af6-12271d7ece4a admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:32.634785 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcca849-ddf5-4b45-8af6-12271d7ece4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:32.636792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcca849-ddf5-4b45-8af6-12271d7ece4a admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:32.657426 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1158/2318] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:45:32 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:32.662116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a03db5-4cb7-4c60-9793-80516c601f15 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:32.672457 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a03db5-4cb7-4c60-9793-80516c601f15 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:32.677180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48973cb5-0a86-49c5-9858-6db698374db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:32.677836 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48973cb5-0a86-49c5-9858-6db698374db9 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:32.678118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48973cb5-0a86-49c5-9858-6db698374db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:32.678390 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48973cb5-0a86-49c5-9858-6db698374db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:32.679151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48973cb5-0a86-49c5-9858-6db698374db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:32.680756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48973cb5-0a86-49c5-9858-6db698374db9 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:32.691473 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1161/2319] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:32.695835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcca849-ddf5-4b45-8af6-12271d7ece4a admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:32.706353 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcca849-ddf5-4b45-8af6-12271d7ece4a admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:32.712353 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6976a8-b906-4f05-a0c0-6b30b6a41e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:32.713254 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6976a8-b906-4f05-a0c0-6b30b6a41e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:32.713537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6976a8-b906-4f05-a0c0-6b30b6a41e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:32.713819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6976a8-b906-4f05-a0c0-6b30b6a41e48 admin admin] PATH_INFO: `/v3/projects/14464fcd692a46f5bd676068da02453d/users/14ae9d3b8c9f47fb8a39024806aab52a/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:32.729479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6976a8-b906-4f05-a0c0-6b30b6a41e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14464fcd692a46f5bd676068da02453d, user_id=14ae9d3b8c9f47fb8a39024806aab52a, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:32.732214 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6976a8-b906-4f05-a0c0-6b30b6a41e48 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:32.746853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1159/2320] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:32 2022] PUT /identity/v3/projects/14464fcd692a46f5bd676068da02453d/users/14ae9d3b8c9f47fb8a39024806aab52a/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:32.750982 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48973cb5-0a86-49c5-9858-6db698374db9 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:32.761272 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48973cb5-0a86-49c5-9858-6db698374db9 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:32.819050 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3606d85-6b58-434d-b25c-c708d597712d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:32.819782 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3606d85-6b58-434d-b25c-c708d597712d admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:32.820083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3606d85-6b58-434d-b25c-c708d597712d admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:32.820334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3606d85-6b58-434d-b25c-c708d597712d admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:32.821058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3606d85-6b58-434d-b25c-c708d597712d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:32.822657 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3606d85-6b58-434d-b25c-c708d597712d admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:32.833889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1162/2321] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:32.838142 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6976a8-b906-4f05-a0c0-6b30b6a41e48 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:32.848718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6976a8-b906-4f05-a0c0-6b30b6a41e48 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:32.855120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25e8d9-bbc8-44e4-be41-da024185b6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:32.856727 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25e8d9-bbc8-44e4-be41-da024185b6d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:32.857006 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25e8d9-bbc8-44e4-be41-da024185b6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:32.857285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25e8d9-bbc8-44e4-be41-da024185b6d6 admin admin] PATH_INFO: `/v3/projects/14464fcd692a46f5bd676068da02453d/users/14ae9d3b8c9f47fb8a39024806aab52a/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:32.862269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25e8d9-bbc8-44e4-be41-da024185b6d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14464fcd692a46f5bd676068da02453d, user_id=14ae9d3b8c9f47fb8a39024806aab52a, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:32.866176 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25e8d9-bbc8-44e4-be41-da024185b6d6 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:32.877693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1160/2322] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:32 2022] PUT /identity/v3/projects/14464fcd692a46f5bd676068da02453d/users/14ae9d3b8c9f47fb8a39024806aab52a/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:32.888139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc063a60-ced0-4b12-ae5d-455c0e6a4051 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:32.888286 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc063a60-ced0-4b12-ae5d-455c0e6a4051 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:32.888889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc063a60-ced0-4b12-ae5d-455c0e6a4051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:32.917680 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-dc063a60-ced0-4b12-ae5d-455c0e6a4051 None None] MFA Rules not processed for user `14ae9d3b8c9f47fb8a39024806aab52a`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:32.918915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-dc063a60-ced0-4b12-ae5d-455c0e6a4051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:32.990151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1163/2323] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:33.001831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25e8d9-bbc8-44e4-be41-da024185b6d6 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:33.003909 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3a25e8d9-bbc8-44e4-be41-da024185b6d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:33.074917 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25e8d9-bbc8-44e4-be41-da024185b6d6 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:33.078980 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194c018-3ce7-4c80-8368-6f7d03716c79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:33.079679 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194c018-3ce7-4c80-8368-6f7d03716c79 service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:33.079965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194c018-3ce7-4c80-8368-6f7d03716c79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:33.080243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194c018-3ce7-4c80-8368-6f7d03716c79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:33.090258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194c018-3ce7-4c80-8368-6f7d03716c79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:33.091752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194c018-3ce7-4c80-8368-6f7d03716c79 service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:33.101835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1161/2324] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:33.169219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc063a60-ced0-4b12-ae5d-455c0e6a4051 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:33.183750 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:33.236934 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc063a60-ced0-4b12-ae5d-455c0e6a4051 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:33.242028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f366895-4bce-4f0d-9052-54c954efbab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:33.242914 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f366895-4bce-4f0d-9052-54c954efbab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:33.243278 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f366895-4bce-4f0d-9052-54c954efbab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:33.243529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f366895-4bce-4f0d-9052-54c954efbab0 admin admin] PATH_INFO: `/v3/users/8a7fc3fd922440219dabcac851223363` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:33.244841 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f366895-4bce-4f0d-9052-54c954efbab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a7fc3fd922440219dabcac851223363)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:33.246756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f366895-4bce-4f0d-9052-54c954efbab0 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:33.297328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-4f366895-4bce-4f0d-9052-54c954efbab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a7fc3fd922440219dabcac851223363'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:33.310526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-4f366895-4bce-4f0d-9052-54c954efbab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a7fc3fd922440219dabcac851223363'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:33.317859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-4f366895-4bce-4f0d-9052-54c954efbab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a7fc3fd922440219dabcac851223363'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:33.318533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-4f366895-4bce-4f0d-9052-54c954efbab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a7fc3fd922440219dabcac851223363'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:33.319200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-4f366895-4bce-4f0d-9052-54c954efbab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a7fc3fd922440219dabcac851223363'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:33.333543 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1164/2325] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:33 2022] DELETE /identity/v3/users/8a7fc3fd922440219dabcac851223363 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:33.338396 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7a875-60e4-449f-a67c-f7958bdef41c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:33.338712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7a875-60e4-449f-a67c-f7958bdef41c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:33.338986 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7a875-60e4-449f-a67c-f7958bdef41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:33.357361 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-ccf7a875-60e4-449f-a67c-f7958bdef41c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:33.358653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ccf7a875-60e4-449f-a67c-f7958bdef41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:33.401451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:33.422767 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1162/2326] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:33.432642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f366895-4bce-4f0d-9052-54c954efbab0 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:33.434132 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4f366895-4bce-4f0d-9052-54c954efbab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:33.493900 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f366895-4bce-4f0d-9052-54c954efbab0 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:33.498268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf2833-f391-44b5-b670-ae1b69e95057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:33.498913 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf2833-f391-44b5-b670-ae1b69e95057 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:33.499221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf2833-f391-44b5-b670-ae1b69e95057 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:33.499499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf2833-f391-44b5-b670-ae1b69e95057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:33.509009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf2833-f391-44b5-b670-ae1b69e95057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:33.510445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf2833-f391-44b5-b670-ae1b69e95057 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:33.520756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1165/2327] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:45:33.804320 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7a875-60e4-449f-a67c-f7958bdef41c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:33.806222 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ccf7a875-60e4-449f-a67c-f7958bdef41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:33.822821 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7a875-60e4-449f-a67c-f7958bdef41c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:33.828257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b439003-ac7a-42cb-8865-876040d1bf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:33.829063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b439003-ac7a-42cb-8865-876040d1bf50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:33.829350 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b439003-ac7a-42cb-8865-876040d1bf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:33.829591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b439003-ac7a-42cb-8865-876040d1bf50 admin admin] PATH_INFO: `/v3/projects/a553b56b6b834dcbaa8540c8c8070cee` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:33.831228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b439003-ac7a-42cb-8865-876040d1bf50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a553b56b6b834dcbaa8540c8c8070cee)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:33.833134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b439003-ac7a-42cb-8865-876040d1bf50 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:33.881403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9b439003-ac7a-42cb-8865-876040d1bf50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a553b56b6b834dcbaa8540c8c8070cee', 'request_id': 'req-9b439003-ac7a-42cb-8865-876040d1bf50'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:33.887541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9b439003-ac7a-42cb-8865-876040d1bf50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a553b56b6b834dcbaa8540c8c8070cee', 'request_id': 'req-9b439003-ac7a-42cb-8865-876040d1bf50'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:33.893295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9b439003-ac7a-42cb-8865-876040d1bf50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a553b56b6b834dcbaa8540c8c8070cee', 'request_id': 'req-9b439003-ac7a-42cb-8865-876040d1bf50'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:33.906781 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9b439003-ac7a-42cb-8865-876040d1bf50 admin admin] The token cache is being invalidate because project a553b56b6b834dcbaa8540c8c8070cee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:33.906993 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9b439003-ac7a-42cb-8865-876040d1bf50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:33.907645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9b439003-ac7a-42cb-8865-876040d1bf50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:33.909252 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1163/2328] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:33 2022] DELETE /identity/v3/projects/a553b56b6b834dcbaa8540c8c8070cee => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:33.913905 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf2833-f391-44b5-b670-ae1b69e95057 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:33.926454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf2833-f391-44b5-b670-ae1b69e95057 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:33.964938 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:33.991343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925da927-0138-4922-b1dc-c772ccea0443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:33.992227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925da927-0138-4922-b1dc-c772ccea0443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:33.992506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925da927-0138-4922-b1dc-c772ccea0443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:33.992776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925da927-0138-4922-b1dc-c772ccea0443 admin admin] PATH_INFO: `/v3/users/14ae9d3b8c9f47fb8a39024806aab52a` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:33.993970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925da927-0138-4922-b1dc-c772ccea0443 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14ae9d3b8c9f47fb8a39024806aab52a)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:33.995953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925da927-0138-4922-b1dc-c772ccea0443 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:34.042707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-925da927-0138-4922-b1dc-c772ccea0443 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14ae9d3b8c9f47fb8a39024806aab52a'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:34.054666 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-925da927-0138-4922-b1dc-c772ccea0443 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14ae9d3b8c9f47fb8a39024806aab52a'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:34.061388 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-925da927-0138-4922-b1dc-c772ccea0443 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14ae9d3b8c9f47fb8a39024806aab52a'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:34.062046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-925da927-0138-4922-b1dc-c772ccea0443 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14ae9d3b8c9f47fb8a39024806aab52a'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:34.062662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-925da927-0138-4922-b1dc-c772ccea0443 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14ae9d3b8c9f47fb8a39024806aab52a'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:34.076290 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1166/2329] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:45:33 2022] DELETE /identity/v3/users/14ae9d3b8c9f47fb8a39024806aab52a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:34.353156 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b439003-ac7a-42cb-8865-876040d1bf50 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:34.354587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9b439003-ac7a-42cb-8865-876040d1bf50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:34.367710 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b439003-ac7a-42cb-8865-876040d1bf50 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:34.427099 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368041c8-886c-4a6b-a2f3-b25ebcf83756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:34.427919 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368041c8-886c-4a6b-a2f3-b25ebcf83756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:34.428258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368041c8-886c-4a6b-a2f3-b25ebcf83756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:34.428529 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368041c8-886c-4a6b-a2f3-b25ebcf83756 admin admin] PATH_INFO: `/v3/projects/14464fcd692a46f5bd676068da02453d` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:34.429829 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368041c8-886c-4a6b-a2f3-b25ebcf83756 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14464fcd692a46f5bd676068da02453d)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:34.432007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368041c8-886c-4a6b-a2f3-b25ebcf83756 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:34.477915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-368041c8-886c-4a6b-a2f3-b25ebcf83756 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14464fcd692a46f5bd676068da02453d', 'request_id': 'req-368041c8-886c-4a6b-a2f3-b25ebcf83756'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:34.483741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-368041c8-886c-4a6b-a2f3-b25ebcf83756 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14464fcd692a46f5bd676068da02453d', 'request_id': 'req-368041c8-886c-4a6b-a2f3-b25ebcf83756'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:34.489139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-368041c8-886c-4a6b-a2f3-b25ebcf83756 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14464fcd692a46f5bd676068da02453d', 'request_id': 'req-368041c8-886c-4a6b-a2f3-b25ebcf83756'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:34.497902 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:34.502855 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-368041c8-886c-4a6b-a2f3-b25ebcf83756 admin admin] The token cache is being invalidate because project 14464fcd692a46f5bd676068da02453d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:45:34.503174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-368041c8-886c-4a6b-a2f3-b25ebcf83756 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:34.504409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-368041c8-886c-4a6b-a2f3-b25ebcf83756 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:45:34.507043 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1164/2330] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:45:34 2022] DELETE /identity/v3/projects/14464fcd692a46f5bd676068da02453d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:34.512091 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36004c-8f09-4962-b871-b9cea1e1d12c None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:34.512385 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36004c-8f09-4962-b871-b9cea1e1d12c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:34.512663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e36004c-8f09-4962-b871-b9cea1e1d12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:34.535478 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-2e36004c-8f09-4962-b871-b9cea1e1d12c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:34.536658 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2e36004c-8f09-4962-b871-b9cea1e1d12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:34.597899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1167/2331] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:34.605563 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254e6d8-7dc9-4d1c-9dc6-d73496533672 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:34.605848 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254e6d8-7dc9-4d1c-9dc6-d73496533672 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:34.606159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254e6d8-7dc9-4d1c-9dc6-d73496533672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:34.624672 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-4254e6d8-7dc9-4d1c-9dc6-d73496533672 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:34.625886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4254e6d8-7dc9-4d1c-9dc6-d73496533672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:34.635099 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1165/2332] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:34.639480 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e36004c-8f09-4962-b871-b9cea1e1d12c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:34.649590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e36004c-8f09-4962-b871-b9cea1e1d12c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:34.654293 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41763a-0c39-446a-afc6-f147ab3bd369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:34.654989 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41763a-0c39-446a-afc6-f147ab3bd369 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:34.655305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41763a-0c39-446a-afc6-f147ab3bd369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:34.655587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41763a-0c39-446a-afc6-f147ab3bd369 admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:34.656353 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41763a-0c39-446a-afc6-f147ab3bd369 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:45:34.656652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41763a-0c39-446a-afc6-f147ab3bd369 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:34.658462 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41763a-0c39-446a-afc6-f147ab3bd369 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:34.671040 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1168/2333] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:45:34.675694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254e6d8-7dc9-4d1c-9dc6-d73496533672 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:34.685270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254e6d8-7dc9-4d1c-9dc6-d73496533672 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:34.690474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25124e2-8545-45ab-be03-da34401a87f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:34.691188 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25124e2-8545-45ab-be03-da34401a87f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:34.691473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25124e2-8545-45ab-be03-da34401a87f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:34.691752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25124e2-8545-45ab-be03-da34401a87f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:34.692446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25124e2-8545-45ab-be03-da34401a87f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:34.694415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25124e2-8545-45ab-be03-da34401a87f3 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:34.720767 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1166/2334] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:45:34 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:45:34.724961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41763a-0c39-446a-afc6-f147ab3bd369 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:34.734923 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41763a-0c39-446a-afc6-f147ab3bd369 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:34.788768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:34.806474 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ebdf4-e000-4b30-bdbd-24967d37818d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:34.807193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ebdf4-e000-4b30-bdbd-24967d37818d admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:34.807479 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ebdf4-e000-4b30-bdbd-24967d37818d admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:34.807754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ebdf4-e000-4b30-bdbd-24967d37818d admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:34.808461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ebdf4-e000-4b30-bdbd-24967d37818d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:34.810404 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ebdf4-e000-4b30-bdbd-24967d37818d admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:34.833646 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1169/2335] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:45:34 2022] POST /identity/v3/users => generated 284 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:45:34.839795 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25124e2-8545-45ab-be03-da34401a87f3 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:34.853128 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25124e2-8545-45ab-be03-da34401a87f3 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:34.858748 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d90af1-9963-4d29-b23c-2585140b3bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:34.859472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d90af1-9963-4d29-b23c-2585140b3bfc admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:34.859752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d90af1-9963-4d29-b23c-2585140b3bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:34.860022 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d90af1-9963-4d29-b23c-2585140b3bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:34.860747 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d90af1-9963-4d29-b23c-2585140b3bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:34.862408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d90af1-9963-4d29-b23c-2585140b3bfc admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:34.873240 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1167/2336] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:34.877334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ebdf4-e000-4b30-bdbd-24967d37818d admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:34.887975 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ebdf4-e000-4b30-bdbd-24967d37818d admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:34.893098 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c651c4-ba3e-491b-8721-95e6146d2821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:34.893750 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c651c4-ba3e-491b-8721-95e6146d2821 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:34.894031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c651c4-ba3e-491b-8721-95e6146d2821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:34.894301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c651c4-ba3e-491b-8721-95e6146d2821 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:34.895045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c651c4-ba3e-491b-8721-95e6146d2821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:34.896664 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c651c4-ba3e-491b-8721-95e6146d2821 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:34.907048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1170/2337] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:34.911470 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d90af1-9963-4d29-b23c-2585140b3bfc admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:34.921899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d90af1-9963-4d29-b23c-2585140b3bfc admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:34.929110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847248d-2848-4339-9eef-4be609f30b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:34.929999 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847248d-2848-4339-9eef-4be609f30b3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:34.930280 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847248d-2848-4339-9eef-4be609f30b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:34.930552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847248d-2848-4339-9eef-4be609f30b3c admin admin] PATH_INFO: `/v3/projects/7dfb371208b74145a20c3d89bd6bf0ca/users/0d54eabc518240f1a2b89757adbfab6a/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:34.945546 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847248d-2848-4339-9eef-4be609f30b3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dfb371208b74145a20c3d89bd6bf0ca, user_id=0d54eabc518240f1a2b89757adbfab6a, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:34.948172 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847248d-2848-4339-9eef-4be609f30b3c admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:34.964699 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1168/2338] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:34 2022] PUT /identity/v3/projects/7dfb371208b74145a20c3d89bd6bf0ca/users/0d54eabc518240f1a2b89757adbfab6a/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:34.981472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805fdbb-bef5-4778-8a55-fe624585d6ed None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:34.981958 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805fdbb-bef5-4778-8a55-fe624585d6ed None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:34.982072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805fdbb-bef5-4778-8a55-fe624585d6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:35.010148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-1805fdbb-bef5-4778-8a55-fe624585d6ed None None] MFA Rules not processed for user `0d54eabc518240f1a2b89757adbfab6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:35.011555 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1805fdbb-bef5-4778-8a55-fe624585d6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:35.077623 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1171/2339] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:35.082560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847248d-2848-4339-9eef-4be609f30b3c admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:35.153008 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847248d-2848-4339-9eef-4be609f30b3c admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:35.158170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91fc4f-1c38-4e0f-ae66-4c3b9bea5f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:35.158887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91fc4f-1c38-4e0f-ae66-4c3b9bea5f55 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:35.159204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91fc4f-1c38-4e0f-ae66-4c3b9bea5f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:35.159941 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91fc4f-1c38-4e0f-ae66-4c3b9bea5f55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:35.160629 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a91fc4f-1c38-4e0f-ae66-4c3b9bea5f55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:35.162549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a91fc4f-1c38-4e0f-ae66-4c3b9bea5f55 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:35.191309 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1169/2340] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:45:35 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:35.192649 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805fdbb-bef5-4778-8a55-fe624585d6ed None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:35.203702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805fdbb-bef5-4778-8a55-fe624585d6ed None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:35.265367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705f986-1c13-46da-81aa-b204e50454f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:35.266055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705f986-1c13-46da-81aa-b204e50454f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:35.266413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705f986-1c13-46da-81aa-b204e50454f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:35.266732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705f986-1c13-46da-81aa-b204e50454f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:35.267506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705f986-1c13-46da-81aa-b204e50454f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:35.269697 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705f986-1c13-46da-81aa-b204e50454f7 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:35.295697 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1172/2341] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:45:35 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:45:35.300215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91fc4f-1c38-4e0f-ae66-4c3b9bea5f55 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:35.312521 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91fc4f-1c38-4e0f-ae66-4c3b9bea5f55 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:35.317424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ce51e-349c-44de-a0ed-e515aeb87f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:35.318100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ce51e-349c-44de-a0ed-e515aeb87f4c admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:35.318387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ce51e-349c-44de-a0ed-e515aeb87f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:35.318666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ce51e-349c-44de-a0ed-e515aeb87f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:35.319490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ce51e-349c-44de-a0ed-e515aeb87f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:35.321205 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ce51e-349c-44de-a0ed-e515aeb87f4c admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:35.332848 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1170/2342] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:35.337214 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705f986-1c13-46da-81aa-b204e50454f7 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:35.347264 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705f986-1c13-46da-81aa-b204e50454f7 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:35.352198 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c7f90-11f6-4fa5-b570-9e1815ca05bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:35.353156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c7f90-11f6-4fa5-b570-9e1815ca05bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:35.353495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c7f90-11f6-4fa5-b570-9e1815ca05bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:35.353777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c7f90-11f6-4fa5-b570-9e1815ca05bf admin admin] PATH_INFO: `/v3/projects/2367bcc629f24c77bb7f185d3f075985/users/a235e38794da4894b301830641ea5e92/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:35.368419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c7f90-11f6-4fa5-b570-9e1815ca05bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2367bcc629f24c77bb7f185d3f075985, user_id=a235e38794da4894b301830641ea5e92, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:35.371151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c7f90-11f6-4fa5-b570-9e1815ca05bf admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:35.384295 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1173/2343] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:35 2022] PUT /identity/v3/projects/2367bcc629f24c77bb7f185d3f075985/users/a235e38794da4894b301830641ea5e92/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:35.388258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ce51e-349c-44de-a0ed-e515aeb87f4c admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:35.398539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ce51e-349c-44de-a0ed-e515aeb87f4c admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:35.457358 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f3b1a-b5d8-4409-b212-7ed2733768d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:35.457950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f3b1a-b5d8-4409-b212-7ed2733768d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:35.458230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f3b1a-b5d8-4409-b212-7ed2733768d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:35.458505 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f3b1a-b5d8-4409-b212-7ed2733768d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:35.459260 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f3b1a-b5d8-4409-b212-7ed2733768d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:35.460911 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f3b1a-b5d8-4409-b212-7ed2733768d2 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:35.470832 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1171/2344] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:45:35.475239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c7f90-11f6-4fa5-b570-9e1815ca05bf admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:35.486425 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c7f90-11f6-4fa5-b570-9e1815ca05bf admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:35.494382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d33746-8bba-4624-9dc3-6534c75ef796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:35.495815 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d33746-8bba-4624-9dc3-6534c75ef796 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:35.496197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d33746-8bba-4624-9dc3-6534c75ef796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:35.496593 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d33746-8bba-4624-9dc3-6534c75ef796 admin admin] PATH_INFO: `/v3/projects/2367bcc629f24c77bb7f185d3f075985/users/a235e38794da4894b301830641ea5e92/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:35.499155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d33746-8bba-4624-9dc3-6534c75ef796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2367bcc629f24c77bb7f185d3f075985, user_id=a235e38794da4894b301830641ea5e92, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:35.501825 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d33746-8bba-4624-9dc3-6534c75ef796 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:35.512773 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1174/2345] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:45:35 2022] PUT /identity/v3/projects/2367bcc629f24c77bb7f185d3f075985/users/a235e38794da4894b301830641ea5e92/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:45:35.525343 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732ec78-d05d-4a28-8ce0-c9adc8392ffb None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:35.525643 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732ec78-d05d-4a28-8ce0-c9adc8392ffb None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:35.525922 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732ec78-d05d-4a28-8ce0-c9adc8392ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:35.532779 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:35.555633 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-8732ec78-d05d-4a28-8ce0-c9adc8392ffb None None] MFA Rules not processed for user `a235e38794da4894b301830641ea5e92`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:45:35.557192 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8732ec78-d05d-4a28-8ce0-c9adc8392ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:35.647656 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1172/2346] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:45:35 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:45:35.655977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d33746-8bba-4624-9dc3-6534c75ef796 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:35.657922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-08d33746-8bba-4624-9dc3-6534c75ef796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:35.664682 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:35.721688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d33746-8bba-4624-9dc3-6534c75ef796 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:35.725979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de5e52-bec6-4baf-80ec-88001f3d245e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:35.726632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de5e52-bec6-4baf-80ec-88001f3d245e service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:35.726918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de5e52-bec6-4baf-80ec-88001f3d245e service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:35.727231 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de5e52-bec6-4baf-80ec-88001f3d245e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:35.738426 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de5e52-bec6-4baf-80ec-88001f3d245e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:35.739910 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de5e52-bec6-4baf-80ec-88001f3d245e service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:35.750156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1175/2347] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:35 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:35.837460 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732ec78-d05d-4a28-8ce0-c9adc8392ffb None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:45:35.839391 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8732ec78-d05d-4a28-8ce0-c9adc8392ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:45:35.852344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732ec78-d05d-4a28-8ce0-c9adc8392ffb None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:45:35.857237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04889e-d97b-4e31-a682-ddbfd9adda70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:45:35.857881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04889e-d97b-4e31-a682-ddbfd9adda70 service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:45:35.858167 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04889e-d97b-4e31-a682-ddbfd9adda70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:45:35.858443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04889e-d97b-4e31-a682-ddbfd9adda70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:45:35.922204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04889e-d97b-4e31-a682-ddbfd9adda70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:45:35.923699 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04889e-d97b-4e31-a682-ddbfd9adda70 service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:45:35.932909 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1173/2348] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:45:35 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:45:45.745333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=25 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:45:45.928872 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=60 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:02.894764 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de5e52-bec6-4baf-80ec-88001f3d245e service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:02.970887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de5e52-bec6-4baf-80ec-88001f3d245e service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:02.975707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d4e21-c31b-4ec2-a04b-d7b120343580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:02.976726 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d4e21-c31b-4ec2-a04b-d7b120343580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:02.977015 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d4e21-c31b-4ec2-a04b-d7b120343580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:02.977439 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d4e21-c31b-4ec2-a04b-d7b120343580 admin admin] PATH_INFO: `/v3/users/0d54eabc518240f1a2b89757adbfab6a` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:02.978662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d4e21-c31b-4ec2-a04b-d7b120343580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d54eabc518240f1a2b89757adbfab6a)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:02.981532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d4e21-c31b-4ec2-a04b-d7b120343580 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:03.048818 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-cd7d4e21-c31b-4ec2-a04b-d7b120343580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d54eabc518240f1a2b89757adbfab6a'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:03.067720 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-cd7d4e21-c31b-4ec2-a04b-d7b120343580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d54eabc518240f1a2b89757adbfab6a'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:03.076858 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-cd7d4e21-c31b-4ec2-a04b-d7b120343580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d54eabc518240f1a2b89757adbfab6a'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:03.077566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-cd7d4e21-c31b-4ec2-a04b-d7b120343580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d54eabc518240f1a2b89757adbfab6a'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:03.078265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-cd7d4e21-c31b-4ec2-a04b-d7b120343580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d54eabc518240f1a2b89757adbfab6a'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:03.098998 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53007b31-d29c-44ab-8850-9abb255bd60b None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:03.099687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53007b31-d29c-44ab-8850-9abb255bd60b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:03.099687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53007b31-d29c-44ab-8850-9abb255bd60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:03.101960 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1176/2349] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:46:02 2022] DELETE /identity/v3/users/0d54eabc518240f1a2b89757adbfab6a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:03.117852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-53007b31-d29c-44ab-8850-9abb255bd60b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:03.119163 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-53007b31-d29c-44ab-8850-9abb255bd60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:03.188510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1174/2350] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:03.199608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d4e21-c31b-4ec2-a04b-d7b120343580 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:03.201036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-cd7d4e21-c31b-4ec2-a04b-d7b120343580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:03.264788 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d4e21-c31b-4ec2-a04b-d7b120343580 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:03.268857 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a392d2-55fa-4722-89ee-6b39dc707768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:03.269802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a392d2-55fa-4722-89ee-6b39dc707768 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:03.270198 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a392d2-55fa-4722-89ee-6b39dc707768 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:03.270583 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a392d2-55fa-4722-89ee-6b39dc707768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:03.282916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a392d2-55fa-4722-89ee-6b39dc707768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:03.284833 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a392d2-55fa-4722-89ee-6b39dc707768 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:03.295315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1177/2351] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:46:03.608691 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53007b31-d29c-44ab-8850-9abb255bd60b None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:03.610297 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-53007b31-d29c-44ab-8850-9abb255bd60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:03.622707 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53007b31-d29c-44ab-8850-9abb255bd60b None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:03.628268 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:03.629238 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:03.629527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:03.630530 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506 admin admin] PATH_INFO: `/v3/projects/7dfb371208b74145a20c3d89bd6bf0ca` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:03.632142 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dfb371208b74145a20c3d89bd6bf0ca)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:03.634281 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:03.685564 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dfb371208b74145a20c3d89bd6bf0ca', 'request_id': 'req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:03.692008 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dfb371208b74145a20c3d89bd6bf0ca', 'request_id': 'req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:03.698918 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dfb371208b74145a20c3d89bd6bf0ca', 'request_id': 'req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:03.713317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506 admin admin] The token cache is being invalidate because project 7dfb371208b74145a20c3d89bd6bf0ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:46:03.713605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:03.714276 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:03.715900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1175/2352] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:46:03 2022] DELETE /identity/v3/projects/7dfb371208b74145a20c3d89bd6bf0ca => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:03.720069 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a392d2-55fa-4722-89ee-6b39dc707768 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:03.731180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a392d2-55fa-4722-89ee-6b39dc707768 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:03.756710 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:03.796811 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9d96f-c54c-4eea-8c4e-825407dc2085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:03.797641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9d96f-c54c-4eea-8c4e-825407dc2085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:03.797924 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9d96f-c54c-4eea-8c4e-825407dc2085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:03.798198 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9d96f-c54c-4eea-8c4e-825407dc2085 admin admin] PATH_INFO: `/v3/users/a235e38794da4894b301830641ea5e92` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:03.799445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9d96f-c54c-4eea-8c4e-825407dc2085 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a235e38794da4894b301830641ea5e92)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:03.801393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9d96f-c54c-4eea-8c4e-825407dc2085 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:03.852429 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8bb9d96f-c54c-4eea-8c4e-825407dc2085 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a235e38794da4894b301830641ea5e92'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:03.864979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8bb9d96f-c54c-4eea-8c4e-825407dc2085 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a235e38794da4894b301830641ea5e92'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:03.871988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8bb9d96f-c54c-4eea-8c4e-825407dc2085 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a235e38794da4894b301830641ea5e92'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:03.872661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8bb9d96f-c54c-4eea-8c4e-825407dc2085 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a235e38794da4894b301830641ea5e92'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:03.873278 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8bb9d96f-c54c-4eea-8c4e-825407dc2085 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a235e38794da4894b301830641ea5e92'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:03.891067 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1178/2353] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:46:03 2022] DELETE /identity/v3/users/a235e38794da4894b301830641ea5e92 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:04.183244 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:04.185237 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:04.205567 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3f034-e780-4dbb-9c23-d0caa7a2f506 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:04.267007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb51b07-9867-4c93-894a-b1bcfdbdaff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:04.267916 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb51b07-9867-4c93-894a-b1bcfdbdaff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:04.268194 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb51b07-9867-4c93-894a-b1bcfdbdaff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:04.268488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb51b07-9867-4c93-894a-b1bcfdbdaff3 admin admin] PATH_INFO: `/v3/projects/2367bcc629f24c77bb7f185d3f075985` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:04.269674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb51b07-9867-4c93-894a-b1bcfdbdaff3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2367bcc629f24c77bb7f185d3f075985)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:04.271788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb51b07-9867-4c93-894a-b1bcfdbdaff3 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:04.304143 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:04.322394 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3bb51b07-9867-4c93-894a-b1bcfdbdaff3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2367bcc629f24c77bb7f185d3f075985', 'request_id': 'req-3bb51b07-9867-4c93-894a-b1bcfdbdaff3'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:04.328798 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3bb51b07-9867-4c93-894a-b1bcfdbdaff3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2367bcc629f24c77bb7f185d3f075985', 'request_id': 'req-3bb51b07-9867-4c93-894a-b1bcfdbdaff3'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:04.334944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3bb51b07-9867-4c93-894a-b1bcfdbdaff3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2367bcc629f24c77bb7f185d3f075985', 'request_id': 'req-3bb51b07-9867-4c93-894a-b1bcfdbdaff3'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:04.348765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3bb51b07-9867-4c93-894a-b1bcfdbdaff3 admin admin] The token cache is being invalidate because project 2367bcc629f24c77bb7f185d3f075985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:46:04.349078 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3bb51b07-9867-4c93-894a-b1bcfdbdaff3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:04.349733 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3bb51b07-9867-4c93-894a-b1bcfdbdaff3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:04.351319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1176/2354] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:46:04 2022] DELETE /identity/v3/projects/2367bcc629f24c77bb7f185d3f075985 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:04.356499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ef67f-064a-48d7-9c13-237dd7aaa877 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:04.356981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ef67f-064a-48d7-9c13-237dd7aaa877 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:04.357097 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ef67f-064a-48d7-9c13-237dd7aaa877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:04.379167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-112ef67f-064a-48d7-9c13-237dd7aaa877 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:04.380337 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-112ef67f-064a-48d7-9c13-237dd7aaa877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:04.442976 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1179/2355] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:04.451857 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a5d70-8e33-4bd1-ab87-0f8bb83f8a5c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:04.452152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a5d70-8e33-4bd1-ab87-0f8bb83f8a5c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:04.452432 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a5d70-8e33-4bd1-ab87-0f8bb83f8a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:04.472070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-9f5a5d70-8e33-4bd1-ab87-0f8bb83f8a5c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:04.473300 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9f5a5d70-8e33-4bd1-ab87-0f8bb83f8a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:04.482676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1177/2356] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:04.487166 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ef67f-064a-48d7-9c13-237dd7aaa877 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:04.497775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ef67f-064a-48d7-9c13-237dd7aaa877 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:04.502905 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891a116-9dc3-4ae2-a3b3-d58a159b6586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:04.503601 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891a116-9dc3-4ae2-a3b3-d58a159b6586 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:04.503886 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891a116-9dc3-4ae2-a3b3-d58a159b6586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:04.504161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891a116-9dc3-4ae2-a3b3-d58a159b6586 admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:04.504910 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891a116-9dc3-4ae2-a3b3-d58a159b6586 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:46:04.505269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891a116-9dc3-4ae2-a3b3-d58a159b6586 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:04.506889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891a116-9dc3-4ae2-a3b3-d58a159b6586 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:04.525224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1180/2357] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:46:04.530039 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a5d70-8e33-4bd1-ab87-0f8bb83f8a5c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:04.540293 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a5d70-8e33-4bd1-ab87-0f8bb83f8a5c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:04.545307 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82add1bf-acef-4f3f-b7f1-cad9a287b719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:04.545990 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82add1bf-acef-4f3f-b7f1-cad9a287b719 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:04.546269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82add1bf-acef-4f3f-b7f1-cad9a287b719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:04.546542 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82add1bf-acef-4f3f-b7f1-cad9a287b719 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:04.547266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82add1bf-acef-4f3f-b7f1-cad9a287b719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:04.549181 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82add1bf-acef-4f3f-b7f1-cad9a287b719 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:04.581964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1178/2358] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:46:04 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:46:04.586501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891a116-9dc3-4ae2-a3b3-d58a159b6586 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:04.599001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891a116-9dc3-4ae2-a3b3-d58a159b6586 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:04.629141 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:04.664129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3ff0f-7e14-442f-b266-2467e1bee3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:04.665541 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3ff0f-7e14-442f-b266-2467e1bee3af admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:04.665541 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3ff0f-7e14-442f-b266-2467e1bee3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:04.665885 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3ff0f-7e14-442f-b266-2467e1bee3af admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:04.666605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3ff0f-7e14-442f-b266-2467e1bee3af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:04.669000 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3ff0f-7e14-442f-b266-2467e1bee3af admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:04.693325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1181/2359] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:46:04 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:46:04.697556 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82add1bf-acef-4f3f-b7f1-cad9a287b719 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:04.708679 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82add1bf-acef-4f3f-b7f1-cad9a287b719 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:04.713880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2b3e7-6e28-4806-86d2-2a73fdf2b22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:04.714539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2b3e7-6e28-4806-86d2-2a73fdf2b22e admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:04.714820 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2b3e7-6e28-4806-86d2-2a73fdf2b22e admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:04.715128 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2b3e7-6e28-4806-86d2-2a73fdf2b22e admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:04.715850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2b3e7-6e28-4806-86d2-2a73fdf2b22e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:04.717457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2b3e7-6e28-4806-86d2-2a73fdf2b22e admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:04.728218 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1179/2360] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:46:04.732230 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3ff0f-7e14-442f-b266-2467e1bee3af admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:04.742449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3ff0f-7e14-442f-b266-2467e1bee3af admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:04.754169 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522293b9-ceb3-4c89-9441-80de415ddff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:04.754862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522293b9-ceb3-4c89-9441-80de415ddff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:04.755179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522293b9-ceb3-4c89-9441-80de415ddff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:04.755568 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522293b9-ceb3-4c89-9441-80de415ddff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:04.756291 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522293b9-ceb3-4c89-9441-80de415ddff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:04.757949 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522293b9-ceb3-4c89-9441-80de415ddff8 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:04.768756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1182/2361] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:46:04.773132 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2b3e7-6e28-4806-86d2-2a73fdf2b22e admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:04.783340 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2b3e7-6e28-4806-86d2-2a73fdf2b22e admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:04.788640 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade00a97-709b-4e40-95d4-3d127422dbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:04.789577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade00a97-709b-4e40-95d4-3d127422dbad admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:04.789866 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade00a97-709b-4e40-95d4-3d127422dbad admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:04.790144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade00a97-709b-4e40-95d4-3d127422dbad admin admin] PATH_INFO: `/v3/projects/abe1c0449c034c2484a9217a7ef93fea/users/2bcae61f2e984751b6c68846fd718a50/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:04.804672 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade00a97-709b-4e40-95d4-3d127422dbad admin admin] RBAC: Authorizing `identity:create_grant(project_id=abe1c0449c034c2484a9217a7ef93fea, user_id=2bcae61f2e984751b6c68846fd718a50, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:04.807461 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade00a97-709b-4e40-95d4-3d127422dbad admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:04.821876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1180/2362] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:46:04 2022] PUT /identity/v3/projects/abe1c0449c034c2484a9217a7ef93fea/users/2bcae61f2e984751b6c68846fd718a50/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:04.832644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f380-16c1-4e4a-871c-49f86783e91b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:04.832878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f380-16c1-4e4a-871c-49f86783e91b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:04.833158 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431f380-16c1-4e4a-871c-49f86783e91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:04.861422 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-e431f380-16c1-4e4a-871c-49f86783e91b None None] MFA Rules not processed for user `2bcae61f2e984751b6c68846fd718a50`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:04.862674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e431f380-16c1-4e4a-871c-49f86783e91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:04.932651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1183/2363] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:04.937278 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade00a97-709b-4e40-95d4-3d127422dbad admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:04.998756 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade00a97-709b-4e40-95d4-3d127422dbad admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:05.003772 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33bc199-6835-4afb-b9dc-a67be5e2dc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:05.004431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bc199-6835-4afb-b9dc-a67be5e2dc62 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:05.004695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bc199-6835-4afb-b9dc-a67be5e2dc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:05.004999 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bc199-6835-4afb-b9dc-a67be5e2dc62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:05.005771 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33bc199-6835-4afb-b9dc-a67be5e2dc62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:05.007750 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33bc199-6835-4afb-b9dc-a67be5e2dc62 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:05.037251 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1181/2364] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:46:04 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:46:05.042869 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f380-16c1-4e4a-871c-49f86783e91b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:05.054109 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431f380-16c1-4e4a-871c-49f86783e91b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:05.114591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b088a-496f-4fe4-9906-af0813decd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:05.115359 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b088a-496f-4fe4-9906-af0813decd45 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:05.115647 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b088a-496f-4fe4-9906-af0813decd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:05.115917 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b088a-496f-4fe4-9906-af0813decd45 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:05.116635 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b088a-496f-4fe4-9906-af0813decd45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:05.118563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b088a-496f-4fe4-9906-af0813decd45 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:05.142545 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1184/2365] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:46:05 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:46:05.143988 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33bc199-6835-4afb-b9dc-a67be5e2dc62 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:05.154294 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33bc199-6835-4afb-b9dc-a67be5e2dc62 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:05.159653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87d971-650d-4d69-babe-b8d8dcef19c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:05.160350 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87d971-650d-4d69-babe-b8d8dcef19c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:05.160624 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87d971-650d-4d69-babe-b8d8dcef19c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:05.160889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87d971-650d-4d69-babe-b8d8dcef19c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:05.161676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc87d971-650d-4d69-babe-b8d8dcef19c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:05.163314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc87d971-650d-4d69-babe-b8d8dcef19c6 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:05.174007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1182/2366] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:46:05.177976 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b088a-496f-4fe4-9906-af0813decd45 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:05.188652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b088a-496f-4fe4-9906-af0813decd45 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:05.194594 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14cc94-e6db-4c06-bfb3-8e1b078c5e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:05.195499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14cc94-e6db-4c06-bfb3-8e1b078c5e6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:05.195775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14cc94-e6db-4c06-bfb3-8e1b078c5e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:05.196049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14cc94-e6db-4c06-bfb3-8e1b078c5e6a admin admin] PATH_INFO: `/v3/projects/b10643783e274469a9c5113dfbb0ba14/users/0e250375bd6944e99c20e76676023d69/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:05.211763 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14cc94-e6db-4c06-bfb3-8e1b078c5e6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10643783e274469a9c5113dfbb0ba14, user_id=0e250375bd6944e99c20e76676023d69, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:05.214472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14cc94-e6db-4c06-bfb3-8e1b078c5e6a admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:05.229122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1185/2367] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:46:05 2022] PUT /identity/v3/projects/b10643783e274469a9c5113dfbb0ba14/users/0e250375bd6944e99c20e76676023d69/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:05.233192 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87d971-650d-4d69-babe-b8d8dcef19c6 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:05.243393 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87d971-650d-4d69-babe-b8d8dcef19c6 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:05.266634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:05.307670 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393c822-e6e0-40ac-82a6-34de4ee227eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:05.308421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393c822-e6e0-40ac-82a6-34de4ee227eb admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:05.308686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393c822-e6e0-40ac-82a6-34de4ee227eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:05.308955 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393c822-e6e0-40ac-82a6-34de4ee227eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:05.309715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393c822-e6e0-40ac-82a6-34de4ee227eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:05.311409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393c822-e6e0-40ac-82a6-34de4ee227eb admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:05.321561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1183/2368] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:46:05.325757 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14cc94-e6db-4c06-bfb3-8e1b078c5e6a admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:05.331762 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:05.338338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14cc94-e6db-4c06-bfb3-8e1b078c5e6a admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:05.344028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2f925-fb9a-4456-990a-51e3b3454e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:05.344924 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2f925-fb9a-4456-990a-51e3b3454e4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:05.345209 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2f925-fb9a-4456-990a-51e3b3454e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:05.345485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2f925-fb9a-4456-990a-51e3b3454e4a admin admin] PATH_INFO: `/v3/projects/b10643783e274469a9c5113dfbb0ba14/users/0e250375bd6944e99c20e76676023d69/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:05.347511 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2f925-fb9a-4456-990a-51e3b3454e4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10643783e274469a9c5113dfbb0ba14, user_id=0e250375bd6944e99c20e76676023d69, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:05.350337 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2f925-fb9a-4456-990a-51e3b3454e4a admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:05.361381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1186/2369] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:46:05 2022] PUT /identity/v3/projects/b10643783e274469a9c5113dfbb0ba14/users/0e250375bd6944e99c20e76676023d69/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:05.372056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bcfc4-e68a-4137-accd-7c2cf8adfde0 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:05.372387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bcfc4-e68a-4137-accd-7c2cf8adfde0 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:05.372641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bcfc4-e68a-4137-accd-7c2cf8adfde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:05.403629 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-d18bcfc4-e68a-4137-accd-7c2cf8adfde0 None None] MFA Rules not processed for user `0e250375bd6944e99c20e76676023d69`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:05.404963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d18bcfc4-e68a-4137-accd-7c2cf8adfde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:05.477585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1184/2370] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:05.489142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2f925-fb9a-4456-990a-51e3b3454e4a admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:05.491289 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-65b2f925-fb9a-4456-990a-51e3b3454e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:05.559438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2f925-fb9a-4456-990a-51e3b3454e4a admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:05.563789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eedfd6-a7f4-4133-a7fc-61e5138339f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:05.564668 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eedfd6-a7f4-4133-a7fc-61e5138339f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:05.565029 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eedfd6-a7f4-4133-a7fc-61e5138339f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:05.565391 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eedfd6-a7f4-4133-a7fc-61e5138339f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:05.630006 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eedfd6-a7f4-4133-a7fc-61e5138339f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:05.631569 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eedfd6-a7f4-4133-a7fc-61e5138339f4 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:05.640437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1187/2371] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:46:09.814214 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bcfc4-e68a-4137-accd-7c2cf8adfde0 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:09.815950 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d18bcfc4-e68a-4137-accd-7c2cf8adfde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:09.889951 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bcfc4-e68a-4137-accd-7c2cf8adfde0 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:09.895531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aae2ae-70bc-4fbb-aa33-a040bc4500ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:09.896680 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aae2ae-70bc-4fbb-aa33-a040bc4500ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:09.897086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aae2ae-70bc-4fbb-aa33-a040bc4500ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:09.897435 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aae2ae-70bc-4fbb-aa33-a040bc4500ff admin admin] PATH_INFO: `/v3/users/d379b2d054df462bb741f8c7d72df7ac` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:09.898707 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aae2ae-70bc-4fbb-aa33-a040bc4500ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=d379b2d054df462bb741f8c7d72df7ac)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:09.900704 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aae2ae-70bc-4fbb-aa33-a040bc4500ff admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:09.951506 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-22aae2ae-70bc-4fbb-aa33-a040bc4500ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd379b2d054df462bb741f8c7d72df7ac'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:09.952669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:09.964199 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-22aae2ae-70bc-4fbb-aa33-a040bc4500ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd379b2d054df462bb741f8c7d72df7ac'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:09.970790 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-22aae2ae-70bc-4fbb-aa33-a040bc4500ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd379b2d054df462bb741f8c7d72df7ac'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:09.971478 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-22aae2ae-70bc-4fbb-aa33-a040bc4500ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd379b2d054df462bb741f8c7d72df7ac'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:09.972069 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-22aae2ae-70bc-4fbb-aa33-a040bc4500ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd379b2d054df462bb741f8c7d72df7ac'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:09.985573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1185/2372] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:46:09 2022] DELETE /identity/v3/users/d379b2d054df462bb741f8c7d72df7ac => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:09.991055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1731d0-5c73-4aeb-977e-62e30701604f None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:09.991588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1731d0-5c73-4aeb-977e-62e30701604f None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:09.991691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1731d0-5c73-4aeb-977e-62e30701604f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:10.010951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-fc1731d0-5c73-4aeb-977e-62e30701604f None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:10.012221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-fc1731d0-5c73-4aeb-977e-62e30701604f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:10.073001 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1188/2373] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:46:10.083064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aae2ae-70bc-4fbb-aa33-a040bc4500ff admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:10.084734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-22aae2ae-70bc-4fbb-aa33-a040bc4500ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:10.143701 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aae2ae-70bc-4fbb-aa33-a040bc4500ff admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:10.147773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c40b9f-468b-451c-821f-2dd99688f4c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:10.148426 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c40b9f-468b-451c-821f-2dd99688f4c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:10.148708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c40b9f-468b-451c-821f-2dd99688f4c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:10.148984 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c40b9f-468b-451c-821f-2dd99688f4c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:10.158513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c40b9f-468b-451c-821f-2dd99688f4c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:10.160373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c40b9f-468b-451c-821f-2dd99688f4c4 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:10.169934 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1186/2374] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:46:10.501368 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1731d0-5c73-4aeb-977e-62e30701604f None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:10.503400 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-fc1731d0-5c73-4aeb-977e-62e30701604f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:10.537513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1731d0-5c73-4aeb-977e-62e30701604f None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:10.543877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a66e99-4883-49fb-ba29-3fc4218f2a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:10.544744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a66e99-4883-49fb-ba29-3fc4218f2a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:10.545119 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a66e99-4883-49fb-ba29-3fc4218f2a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:10.545449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a66e99-4883-49fb-ba29-3fc4218f2a2d admin admin] PATH_INFO: `/v3/projects/fdd73263047847afa95bac79ae5fdf0a` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:10.546677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a66e99-4883-49fb-ba29-3fc4218f2a2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdd73263047847afa95bac79ae5fdf0a)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:10.548741 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a66e99-4883-49fb-ba29-3fc4218f2a2d admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:10.598527 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:10.606241 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-29a66e99-4883-49fb-ba29-3fc4218f2a2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdd73263047847afa95bac79ae5fdf0a', 'request_id': 'req-29a66e99-4883-49fb-ba29-3fc4218f2a2d'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:10.614393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-29a66e99-4883-49fb-ba29-3fc4218f2a2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdd73263047847afa95bac79ae5fdf0a', 'request_id': 'req-29a66e99-4883-49fb-ba29-3fc4218f2a2d'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:10.623120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-29a66e99-4883-49fb-ba29-3fc4218f2a2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdd73263047847afa95bac79ae5fdf0a', 'request_id': 'req-29a66e99-4883-49fb-ba29-3fc4218f2a2d'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:10.636971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-29a66e99-4883-49fb-ba29-3fc4218f2a2d admin admin] The token cache is being invalidate because project fdd73263047847afa95bac79ae5fdf0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:46:10.637302 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-29a66e99-4883-49fb-ba29-3fc4218f2a2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:10.638118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-29a66e99-4883-49fb-ba29-3fc4218f2a2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:10.639728 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1189/2375] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:46:10 2022] DELETE /identity/v3/projects/fdd73263047847afa95bac79ae5fdf0a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:12.013886 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c40b9f-468b-451c-821f-2dd99688f4c4 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:12.015674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-08c40b9f-468b-451c-821f-2dd99688f4c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:12.036965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c40b9f-468b-451c-821f-2dd99688f4c4 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:12.118089 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ef362-a644-4ae4-85cb-b00793852a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:12.119029 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ef362-a644-4ae4-85cb-b00793852a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:12.119324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ef362-a644-4ae4-85cb-b00793852a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:12.119599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ef362-a644-4ae4-85cb-b00793852a82 admin admin] PATH_INFO: `/v3/users/b7ed8a1b6122461ab40095df98473d6c` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:12.120931 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ef362-a644-4ae4-85cb-b00793852a82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7ed8a1b6122461ab40095df98473d6c)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:12.122923 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ef362-a644-4ae4-85cb-b00793852a82 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:12.171314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b29ef362-a644-4ae4-85cb-b00793852a82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7ed8a1b6122461ab40095df98473d6c'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:12.183587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b29ef362-a644-4ae4-85cb-b00793852a82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7ed8a1b6122461ab40095df98473d6c'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:12.188135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:12.191414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b29ef362-a644-4ae4-85cb-b00793852a82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7ed8a1b6122461ab40095df98473d6c'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:12.192307 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b29ef362-a644-4ae4-85cb-b00793852a82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7ed8a1b6122461ab40095df98473d6c'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:12.193456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b29ef362-a644-4ae4-85cb-b00793852a82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7ed8a1b6122461ab40095df98473d6c'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:12.210814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1187/2376] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:46:12 2022] DELETE /identity/v3/users/b7ed8a1b6122461ab40095df98473d6c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:12.212306 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd40ae9-b76d-4ece-944b-c397ad6956e3 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:12.212844 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd40ae9-b76d-4ece-944b-c397ad6956e3 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:12.212945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd40ae9-b76d-4ece-944b-c397ad6956e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:12.232599 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-1fd40ae9-b76d-4ece-944b-c397ad6956e3 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:12.234165 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1fd40ae9-b76d-4ece-944b-c397ad6956e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:12.311883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1190/2377] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:46:12.321867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ef362-a644-4ae4-85cb-b00793852a82 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:12.323363 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b29ef362-a644-4ae4-85cb-b00793852a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:12.385621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ef362-a644-4ae4-85cb-b00793852a82 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:12.389521 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73943240-959a-4d1e-b49e-505cd734f8f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:12.390177 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73943240-959a-4d1e-b49e-505cd734f8f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:12.390458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73943240-959a-4d1e-b49e-505cd734f8f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:12.390726 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73943240-959a-4d1e-b49e-505cd734f8f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:12.400163 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73943240-959a-4d1e-b49e-505cd734f8f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:12.401696 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73943240-959a-4d1e-b49e-505cd734f8f1 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:12.411522 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1188/2378] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:46:12.761883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd40ae9-b76d-4ece-944b-c397ad6956e3 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:12.764061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1fd40ae9-b76d-4ece-944b-c397ad6956e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:12.776096 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd40ae9-b76d-4ece-944b-c397ad6956e3 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:12.781759 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e0b62-0248-4274-a7d2-f8598be094ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:12.782846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e0b62-0248-4274-a7d2-f8598be094ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:12.783555 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e0b62-0248-4274-a7d2-f8598be094ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:12.783555 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e0b62-0248-4274-a7d2-f8598be094ae admin admin] PATH_INFO: `/v3/projects/7949453e518b4f11818d4b43a6857321` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:12.785425 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e0b62-0248-4274-a7d2-f8598be094ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=7949453e518b4f11818d4b43a6857321)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:12.787885 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e0b62-0248-4274-a7d2-f8598be094ae admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:12.842580 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-bd3e0b62-0248-4274-a7d2-f8598be094ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7949453e518b4f11818d4b43a6857321', 'request_id': 'req-bd3e0b62-0248-4274-a7d2-f8598be094ae'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:12.848498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-bd3e0b62-0248-4274-a7d2-f8598be094ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7949453e518b4f11818d4b43a6857321', 'request_id': 'req-bd3e0b62-0248-4274-a7d2-f8598be094ae'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:12.856282 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-bd3e0b62-0248-4274-a7d2-f8598be094ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7949453e518b4f11818d4b43a6857321', 'request_id': 'req-bd3e0b62-0248-4274-a7d2-f8598be094ae'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:12.868988 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-bd3e0b62-0248-4274-a7d2-f8598be094ae admin admin] The token cache is being invalidate because project 7949453e518b4f11818d4b43a6857321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:46:12.869385 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-bd3e0b62-0248-4274-a7d2-f8598be094ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:12.870095 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-bd3e0b62-0248-4274-a7d2-f8598be094ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:12.871818 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1191/2379] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:46:12 2022] DELETE /identity/v3/projects/7949453e518b4f11818d4b43a6857321 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:12.877346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989b318-89b7-4643-b4f8-7eb22f1a4df0 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:12.877668 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989b318-89b7-4643-b4f8-7eb22f1a4df0 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:12.877913 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989b318-89b7-4643-b4f8-7eb22f1a4df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:12.896093 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-8989b318-89b7-4643-b4f8-7eb22f1a4df0 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:12.897346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8989b318-89b7-4643-b4f8-7eb22f1a4df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:12.965790 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1189/2380] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:46:12.974365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e05c9-7c3c-4ec7-954b-66b1bc0645a2 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:12.974724 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e05c9-7c3c-4ec7-954b-66b1bc0645a2 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:12.975030 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e05c9-7c3c-4ec7-954b-66b1bc0645a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:12.993919 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-e94e05c9-7c3c-4ec7-954b-66b1bc0645a2 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:12.995340 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e94e05c9-7c3c-4ec7-954b-66b1bc0645a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:13.009688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1192/2381] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:13.015028 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989b318-89b7-4643-b4f8-7eb22f1a4df0 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:13.026485 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989b318-89b7-4643-b4f8-7eb22f1a4df0 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:13.038257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089692d-03d1-4040-bf26-0f67e5d65bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:13.039821 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089692d-03d1-4040-bf26-0f67e5d65bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:13.040348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089692d-03d1-4040-bf26-0f67e5d65bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:13.040824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089692d-03d1-4040-bf26-0f67e5d65bb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:13.041797 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089692d-03d1-4040-bf26-0f67e5d65bb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:46:13.042122 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089692d-03d1-4040-bf26-0f67e5d65bb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:13.044619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089692d-03d1-4040-bf26-0f67e5d65bb7 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:13.062023 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1190/2382] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:46:13.076282 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e05c9-7c3c-4ec7-954b-66b1bc0645a2 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:13.092550 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e05c9-7c3c-4ec7-954b-66b1bc0645a2 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:13.098648 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770baa53-2ba2-4209-ad36-6cebad72ee28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:13.099351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770baa53-2ba2-4209-ad36-6cebad72ee28 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:13.099684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770baa53-2ba2-4209-ad36-6cebad72ee28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:13.099961 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770baa53-2ba2-4209-ad36-6cebad72ee28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:13.100806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770baa53-2ba2-4209-ad36-6cebad72ee28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:13.103291 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770baa53-2ba2-4209-ad36-6cebad72ee28 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:13.148117 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089692d-03d1-4040-bf26-0f67e5d65bb7 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:13.149182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1193/2383] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:46:13 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:46:13.152673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4089692d-03d1-4040-bf26-0f67e5d65bb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:13.154185 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770baa53-2ba2-4209-ad36-6cebad72ee28 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:13.201653 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:13.247445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089692d-03d1-4040-bf26-0f67e5d65bb7 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:13.256959 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770baa53-2ba2-4209-ad36-6cebad72ee28 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:13.261803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b24ef3-28c8-444b-a93a-2922182be5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:13.262709 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b24ef3-28c8-444b-a93a-2922182be5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:13.263004 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b24ef3-28c8-444b-a93a-2922182be5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:13.263308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b24ef3-28c8-444b-a93a-2922182be5d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:13.264011 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b24ef3-28c8-444b-a93a-2922182be5d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:13.265971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b24ef3-28c8-444b-a93a-2922182be5d9 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:13.292633 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1194/2384] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:46:13 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:46:13.293393 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:13.297756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b24ef3-28c8-444b-a93a-2922182be5d9 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:13.311297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b24ef3-28c8-444b-a93a-2922182be5d9 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:13.317613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a2352-60a6-4196-b1ba-ae67ad1440f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:13.318655 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a2352-60a6-4196-b1ba-ae67ad1440f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:13.319795 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a2352-60a6-4196-b1ba-ae67ad1440f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:13.320218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a2352-60a6-4196-b1ba-ae67ad1440f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:13.321981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a2352-60a6-4196-b1ba-ae67ad1440f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:13.322876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505db8c-42e9-48da-9e15-99fc45e672ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:13.323794 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505db8c-42e9-48da-9e15-99fc45e672ff service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:13.323917 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505db8c-42e9-48da-9e15-99fc45e672ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:13.324261 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505db8c-42e9-48da-9e15-99fc45e672ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:13.326106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-c505db8c-42e9-48da-9e15-99fc45e672ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:13.331840 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a2352-60a6-4196-b1ba-ae67ad1440f1 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:13.348525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1195/2385] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:46:13.352742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a2352-60a6-4196-b1ba-ae67ad1440f1 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:13.364369 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a2352-60a6-4196-b1ba-ae67ad1440f1 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:13.370612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae235d5c-b3ee-4c02-8f6d-52d272292018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:13.371590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae235d5c-b3ee-4c02-8f6d-52d272292018 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:13.371934 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae235d5c-b3ee-4c02-8f6d-52d272292018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:13.372222 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae235d5c-b3ee-4c02-8f6d-52d272292018 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:13.373139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae235d5c-b3ee-4c02-8f6d-52d272292018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:13.374919 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae235d5c-b3ee-4c02-8f6d-52d272292018 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:13.386125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1196/2386] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:46:13.390328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae235d5c-b3ee-4c02-8f6d-52d272292018 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:13.402326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae235d5c-b3ee-4c02-8f6d-52d272292018 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:13.407862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00302e-dabb-443f-8b67-b1929e22a7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:13.408793 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00302e-dabb-443f-8b67-b1929e22a7f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:13.409066 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00302e-dabb-443f-8b67-b1929e22a7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:13.409731 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00302e-dabb-443f-8b67-b1929e22a7f0 admin admin] PATH_INFO: `/v3/projects/1f1312cda60542e7a33e36dadd22dc06/users/0f12af5e4f57430599a18f99162b3159/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:13.414804 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505db8c-42e9-48da-9e15-99fc45e672ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:13.416301 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505db8c-42e9-48da-9e15-99fc45e672ff service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:13.425276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00302e-dabb-443f-8b67-b1929e22a7f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f1312cda60542e7a33e36dadd22dc06, user_id=0f12af5e4f57430599a18f99162b3159, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:13.427722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1191/2387] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:46:13.428342 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00302e-dabb-443f-8b67-b1929e22a7f0 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:13.447091 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1197/2388] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:46:13 2022] PUT /identity/v3/projects/1f1312cda60542e7a33e36dadd22dc06/users/0f12af5e4f57430599a18f99162b3159/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:46:13.456309 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28d09a-6933-4698-a7c6-11433b62c85e None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:13.457075 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28d09a-6933-4698-a7c6-11433b62c85e None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:13.457360 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28d09a-6933-4698-a7c6-11433b62c85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:13.482304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970307a8-e3d6-429e-b5b3-d58f1de215ae None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:13.482595 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970307a8-e3d6-429e-b5b3-d58f1de215ae None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:13.482876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970307a8-e3d6-429e-b5b3-d58f1de215ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:13.492451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-9e28d09a-6933-4698-a7c6-11433b62c85e None None] MFA Rules not processed for user `0f12af5e4f57430599a18f99162b3159`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:13.493887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9e28d09a-6933-4698-a7c6-11433b62c85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:13.499812 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-970307a8-e3d6-429e-b5b3-d58f1de215ae None None] MFA Rules not processed for user `563f1367093e493bae87a21374a3c2f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:13.501526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-970307a8-e3d6-429e-b5b3-d58f1de215ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:13.578632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1192/2389] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:13.580441 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1198/2390] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:46:13.586721 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28d09a-6933-4698-a7c6-11433b62c85e None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:13.588953 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9e28d09a-6933-4698-a7c6-11433b62c85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:13.654981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28d09a-6933-4698-a7c6-11433b62c85e None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:13.664503 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d180b-862b-46fb-a0b5-3913a1c902df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:13.665323 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d180b-862b-46fb-a0b5-3913a1c902df service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:13.665992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d180b-862b-46fb-a0b5-3913a1c902df service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:13.665992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d180b-862b-46fb-a0b5-3913a1c902df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:13.681011 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d180b-862b-46fb-a0b5-3913a1c902df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:13.683433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d180b-862b-46fb-a0b5-3913a1c902df service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:13.697568 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1193/2391] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:46:14.480598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970307a8-e3d6-429e-b5b3-d58f1de215ae None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:14.482170 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-970307a8-e3d6-429e-b5b3-d58f1de215ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:14.557871 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970307a8-e3d6-429e-b5b3-d58f1de215ae None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:14.562522 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c7c97-560b-4bed-92cb-b97e15e053f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:14.563437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c7c97-560b-4bed-92cb-b97e15e053f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:14.563724 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c7c97-560b-4bed-92cb-b97e15e053f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:14.563990 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c7c97-560b-4bed-92cb-b97e15e053f1 admin admin] PATH_INFO: `/v3/users/45198a704d7c4ff79c21d10cf2057074` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:14.565191 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c7c97-560b-4bed-92cb-b97e15e053f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45198a704d7c4ff79c21d10cf2057074)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:14.567208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c7c97-560b-4bed-92cb-b97e15e053f1 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:14.581094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:14.621508 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e66c7c97-560b-4bed-92cb-b97e15e053f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45198a704d7c4ff79c21d10cf2057074'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:14.635469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e66c7c97-560b-4bed-92cb-b97e15e053f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45198a704d7c4ff79c21d10cf2057074'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:14.642432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e66c7c97-560b-4bed-92cb-b97e15e053f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45198a704d7c4ff79c21d10cf2057074'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:14.643165 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e66c7c97-560b-4bed-92cb-b97e15e053f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45198a704d7c4ff79c21d10cf2057074'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:14.643772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-e66c7c97-560b-4bed-92cb-b97e15e053f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45198a704d7c4ff79c21d10cf2057074'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:14.658503 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1199/2392] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:46:14 2022] DELETE /identity/v3/users/45198a704d7c4ff79c21d10cf2057074 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:14.664003 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7f8cc-0a62-4b12-8ded-3d3c1b979b04 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:14.664445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7f8cc-0a62-4b12-8ded-3d3c1b979b04 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:14.664764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7f8cc-0a62-4b12-8ded-3d3c1b979b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:14.683045 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-37c7f8cc-0a62-4b12-8ded-3d3c1b979b04 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:14.684274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-37c7f8cc-0a62-4b12-8ded-3d3c1b979b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:14.694630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:14.760541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1194/2393] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:14.760928 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c7c97-560b-4bed-92cb-b97e15e053f1 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:14.763231 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e66c7c97-560b-4bed-92cb-b97e15e053f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:14.831684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c7c97-560b-4bed-92cb-b97e15e053f1 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:14.836656 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427fa66-7296-4c12-9946-f0fd1c0a60c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:14.836656 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427fa66-7296-4c12-9946-f0fd1c0a60c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:14.836908 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427fa66-7296-4c12-9946-f0fd1c0a60c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:14.836993 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427fa66-7296-4c12-9946-f0fd1c0a60c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:14.847399 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427fa66-7296-4c12-9946-f0fd1c0a60c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:14.860993 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427fa66-7296-4c12-9946-f0fd1c0a60c0 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:14.860993 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1200/2394] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:46:15.152277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7f8cc-0a62-4b12-8ded-3d3c1b979b04 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:15.153793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-37c7f8cc-0a62-4b12-8ded-3d3c1b979b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:15.166374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7f8cc-0a62-4b12-8ded-3d3c1b979b04 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:15.184339 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a659c-42a0-43b9-b4aa-18a2f5efb3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:15.185556 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a659c-42a0-43b9-b4aa-18a2f5efb3a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:15.185832 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a659c-42a0-43b9-b4aa-18a2f5efb3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:15.186106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a659c-42a0-43b9-b4aa-18a2f5efb3a0 admin admin] PATH_INFO: `/v3/projects/3f2cc047ebb946e78ea556dfb0ca3468` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:15.187412 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a659c-42a0-43b9-b4aa-18a2f5efb3a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f2cc047ebb946e78ea556dfb0ca3468)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:15.189333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a659c-42a0-43b9-b4aa-18a2f5efb3a0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:15.234729 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-cf7a659c-42a0-43b9-b4aa-18a2f5efb3a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f2cc047ebb946e78ea556dfb0ca3468', 'request_id': 'req-cf7a659c-42a0-43b9-b4aa-18a2f5efb3a0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:15.240967 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-cf7a659c-42a0-43b9-b4aa-18a2f5efb3a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f2cc047ebb946e78ea556dfb0ca3468', 'request_id': 'req-cf7a659c-42a0-43b9-b4aa-18a2f5efb3a0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:15.246930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-cf7a659c-42a0-43b9-b4aa-18a2f5efb3a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f2cc047ebb946e78ea556dfb0ca3468', 'request_id': 'req-cf7a659c-42a0-43b9-b4aa-18a2f5efb3a0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:15.260555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-cf7a659c-42a0-43b9-b4aa-18a2f5efb3a0 admin admin] The token cache is being invalidate because project 3f2cc047ebb946e78ea556dfb0ca3468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:46:15.260834 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-cf7a659c-42a0-43b9-b4aa-18a2f5efb3a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:15.261460 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-cf7a659c-42a0-43b9-b4aa-18a2f5efb3a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:15.263120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1195/2395] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:46:15 2022] DELETE /identity/v3/projects/3f2cc047ebb946e78ea556dfb0ca3468 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:15.268188 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85db49-fbea-4d44-b701-a1f50cfce729 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:15.268503 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85db49-fbea-4d44-b701-a1f50cfce729 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:15.268793 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b85db49-fbea-4d44-b701-a1f50cfce729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:15.287608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-8b85db49-fbea-4d44-b701-a1f50cfce729 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:15.288832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8b85db49-fbea-4d44-b701-a1f50cfce729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:15.366240 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d979af-f4c4-4372-9c71-2cc05fb19933 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:15.366969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d979af-f4c4-4372-9c71-2cc05fb19933 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:15.367121 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d979af-f4c4-4372-9c71-2cc05fb19933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:15.369360 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1201/2396] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:15.391776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-69d979af-f4c4-4372-9c71-2cc05fb19933 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:15.393158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-69d979af-f4c4-4372-9c71-2cc05fb19933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:15.405757 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1196/2397] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:15.410848 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85db49-fbea-4d44-b701-a1f50cfce729 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:15.423433 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b85db49-fbea-4d44-b701-a1f50cfce729 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:15.428803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c5069-170f-4926-bb29-d34df933c919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:15.429567 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c5069-170f-4926-bb29-d34df933c919 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:15.429906 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c5069-170f-4926-bb29-d34df933c919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:15.430142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c5069-170f-4926-bb29-d34df933c919 admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:15.431800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c5069-170f-4926-bb29-d34df933c919 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:46:15.431800 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c5069-170f-4926-bb29-d34df933c919 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:15.433014 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c5069-170f-4926-bb29-d34df933c919 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:15.453374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d979af-f4c4-4372-9c71-2cc05fb19933 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:15.459377 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1202/2398] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:46:15.463110 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d979af-f4c4-4372-9c71-2cc05fb19933 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:15.469386 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba7876-1209-4d59-a279-39a40cdac367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:15.470695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba7876-1209-4d59-a279-39a40cdac367 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:15.470981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba7876-1209-4d59-a279-39a40cdac367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:15.471294 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba7876-1209-4d59-a279-39a40cdac367 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:15.471998 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba7876-1209-4d59-a279-39a40cdac367 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:15.474574 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba7876-1209-4d59-a279-39a40cdac367 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:15.505296 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1197/2399] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:46:15 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:46:15.509736 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c5069-170f-4926-bb29-d34df933c919 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:15.520735 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c5069-170f-4926-bb29-d34df933c919 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:15.541640 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:15.589883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccea11-be87-4efd-86d7-69ee8fb075ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:15.590818 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccea11-be87-4efd-86d7-69ee8fb075ae admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:15.591212 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccea11-be87-4efd-86d7-69ee8fb075ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:15.591555 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ccea11-be87-4efd-86d7-69ee8fb075ae admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:15.592343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccea11-be87-4efd-86d7-69ee8fb075ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:15.594363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ccea11-be87-4efd-86d7-69ee8fb075ae admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:15.615767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1203/2400] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:46:15 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:46:15.620037 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba7876-1209-4d59-a279-39a40cdac367 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:15.630476 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba7876-1209-4d59-a279-39a40cdac367 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:15.635163 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98b82e-b8b6-4644-9723-4c1b60c8f1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:15.635852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98b82e-b8b6-4644-9723-4c1b60c8f1f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:15.636135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98b82e-b8b6-4644-9723-4c1b60c8f1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:15.636411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98b82e-b8b6-4644-9723-4c1b60c8f1f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:15.637140 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98b82e-b8b6-4644-9723-4c1b60c8f1f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:15.638834 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98b82e-b8b6-4644-9723-4c1b60c8f1f9 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:15.649614 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1198/2401] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:46:15.653551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccea11-be87-4efd-86d7-69ee8fb075ae admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:15.664592 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ccea11-be87-4efd-86d7-69ee8fb075ae admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:15.669996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35026c4a-e20b-4783-b1cc-7430878c19e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:15.670690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35026c4a-e20b-4783-b1cc-7430878c19e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:15.671244 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35026c4a-e20b-4783-b1cc-7430878c19e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:15.671379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35026c4a-e20b-4783-b1cc-7430878c19e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:15.672414 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35026c4a-e20b-4783-b1cc-7430878c19e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:15.673937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35026c4a-e20b-4783-b1cc-7430878c19e0 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:15.684962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1204/2402] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:46:15.689065 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98b82e-b8b6-4644-9723-4c1b60c8f1f9 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:15.700492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98b82e-b8b6-4644-9723-4c1b60c8f1f9 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:15.706585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f668daa-ee22-460f-b356-3338d847f332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:15.708165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f668daa-ee22-460f-b356-3338d847f332 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:15.708447 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f668daa-ee22-460f-b356-3338d847f332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:15.708735 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f668daa-ee22-460f-b356-3338d847f332 admin admin] PATH_INFO: `/v3/projects/92141c036fca417cbc3f4a7ebad30700/users/d1ccedf6fb8749ffb1773670c2041fe7/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:15.725295 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f668daa-ee22-460f-b356-3338d847f332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92141c036fca417cbc3f4a7ebad30700, user_id=d1ccedf6fb8749ffb1773670c2041fe7, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:15.728040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f668daa-ee22-460f-b356-3338d847f332 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:15.743357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1199/2403] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:46:15 2022] PUT /identity/v3/projects/92141c036fca417cbc3f4a7ebad30700/users/d1ccedf6fb8749ffb1773670c2041fe7/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:15.753936 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243b19f-ddde-4b2b-afcf-8e79c2d3c3e8 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:15.754223 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243b19f-ddde-4b2b-afcf-8e79c2d3c3e8 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:15.754497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243b19f-ddde-4b2b-afcf-8e79c2d3c3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:15.782892 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-a243b19f-ddde-4b2b-afcf-8e79c2d3c3e8 None None] MFA Rules not processed for user `d1ccedf6fb8749ffb1773670c2041fe7`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:15.784595 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a243b19f-ddde-4b2b-afcf-8e79c2d3c3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:15.853714 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1205/2404] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:15.864406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f668daa-ee22-460f-b356-3338d847f332 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:15.866216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5f668daa-ee22-460f-b356-3338d847f332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:15.907135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:15.926675 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f668daa-ee22-460f-b356-3338d847f332 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:15.930905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9300ec7-02df-4a25-9fb1-10d149600175 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:15.931642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9300ec7-02df-4a25-9fb1-10d149600175 service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:15.931923 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9300ec7-02df-4a25-9fb1-10d149600175 service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:15.932195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9300ec7-02df-4a25-9fb1-10d149600175 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:15.941315 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9300ec7-02df-4a25-9fb1-10d149600175 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:15.942695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9300ec7-02df-4a25-9fb1-10d149600175 service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:15.952215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1200/2405] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:46:17.478034 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243b19f-ddde-4b2b-afcf-8e79c2d3c3e8 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:17.479620 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a243b19f-ddde-4b2b-afcf-8e79c2d3c3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:17.588730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243b19f-ddde-4b2b-afcf-8e79c2d3c3e8 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:17.595242 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695a8a4-472c-4f35-a605-4d5e21bbe45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:17.596851 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695a8a4-472c-4f35-a605-4d5e21bbe45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:17.597405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695a8a4-472c-4f35-a605-4d5e21bbe45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:17.597837 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695a8a4-472c-4f35-a605-4d5e21bbe45d admin admin] PATH_INFO: `/v3/users/0f12af5e4f57430599a18f99162b3159` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:17.600111 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695a8a4-472c-4f35-a605-4d5e21bbe45d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f12af5e4f57430599a18f99162b3159)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:17.602082 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695a8a4-472c-4f35-a605-4d5e21bbe45d admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:17.640920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:17.661197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b695a8a4-472c-4f35-a605-4d5e21bbe45d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f12af5e4f57430599a18f99162b3159'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:17.674969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b695a8a4-472c-4f35-a605-4d5e21bbe45d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f12af5e4f57430599a18f99162b3159'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:17.683482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b695a8a4-472c-4f35-a605-4d5e21bbe45d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f12af5e4f57430599a18f99162b3159'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:17.684370 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b695a8a4-472c-4f35-a605-4d5e21bbe45d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f12af5e4f57430599a18f99162b3159'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:17.685009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b695a8a4-472c-4f35-a605-4d5e21bbe45d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f12af5e4f57430599a18f99162b3159'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:17.701984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1206/2406] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:46:17 2022] DELETE /identity/v3/users/0f12af5e4f57430599a18f99162b3159 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:17.706994 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a65026-7728-4b7b-af0b-0ec222ed0ba4 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:17.707897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a65026-7728-4b7b-af0b-0ec222ed0ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:17.708207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a65026-7728-4b7b-af0b-0ec222ed0ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:17.726391 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-57a65026-7728-4b7b-af0b-0ec222ed0ba4 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:17.727708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-57a65026-7728-4b7b-af0b-0ec222ed0ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:17.794248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1201/2407] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:17.804515 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695a8a4-472c-4f35-a605-4d5e21bbe45d admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:17.806034 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b695a8a4-472c-4f35-a605-4d5e21bbe45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:17.890396 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695a8a4-472c-4f35-a605-4d5e21bbe45d admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:17.895693 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092574d-55cb-413c-82c6-ad088d808996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:17.896455 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092574d-55cb-413c-82c6-ad088d808996 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:17.896750 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092574d-55cb-413c-82c6-ad088d808996 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:17.897034 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092574d-55cb-413c-82c6-ad088d808996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:17.911346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092574d-55cb-413c-82c6-ad088d808996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:17.915339 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092574d-55cb-413c-82c6-ad088d808996 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:17.924869 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1207/2408] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:46:18.215489 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a65026-7728-4b7b-af0b-0ec222ed0ba4 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:18.217000 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-57a65026-7728-4b7b-af0b-0ec222ed0ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:18.237250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a65026-7728-4b7b-af0b-0ec222ed0ba4 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:18.244496 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4e130-b98d-416d-8d14-1f49c2074cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:18.245403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4e130-b98d-416d-8d14-1f49c2074cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:18.245660 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4e130-b98d-416d-8d14-1f49c2074cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:18.246029 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4e130-b98d-416d-8d14-1f49c2074cd4 admin admin] PATH_INFO: `/v3/projects/1f1312cda60542e7a33e36dadd22dc06` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:18.247419 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4e130-b98d-416d-8d14-1f49c2074cd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f1312cda60542e7a33e36dadd22dc06)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:18.250269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4e130-b98d-416d-8d14-1f49c2074cd4 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:18.303631 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b2e4e130-b98d-416d-8d14-1f49c2074cd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f1312cda60542e7a33e36dadd22dc06', 'request_id': 'req-b2e4e130-b98d-416d-8d14-1f49c2074cd4'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:18.309885 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b2e4e130-b98d-416d-8d14-1f49c2074cd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f1312cda60542e7a33e36dadd22dc06', 'request_id': 'req-b2e4e130-b98d-416d-8d14-1f49c2074cd4'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:18.315982 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b2e4e130-b98d-416d-8d14-1f49c2074cd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f1312cda60542e7a33e36dadd22dc06', 'request_id': 'req-b2e4e130-b98d-416d-8d14-1f49c2074cd4'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:18.329368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b2e4e130-b98d-416d-8d14-1f49c2074cd4 admin admin] The token cache is being invalidate because project 1f1312cda60542e7a33e36dadd22dc06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:46:18.329658 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b2e4e130-b98d-416d-8d14-1f49c2074cd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:18.330290 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b2e4e130-b98d-416d-8d14-1f49c2074cd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:18.332277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1202/2409] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:46:18 2022] DELETE /identity/v3/projects/1f1312cda60542e7a33e36dadd22dc06 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:18.337414 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06518204-8f9a-4f62-aa2f-bae2abd73a64 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:18.337707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06518204-8f9a-4f62-aa2f-bae2abd73a64 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:18.337980 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06518204-8f9a-4f62-aa2f-bae2abd73a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:18.359576 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-06518204-8f9a-4f62-aa2f-bae2abd73a64 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:18.360870 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-06518204-8f9a-4f62-aa2f-bae2abd73a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:18.428923 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1208/2410] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:18.436990 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03844b33-5234-49dc-bfe5-973485d15c6e None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:18.437278 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03844b33-5234-49dc-bfe5-973485d15c6e None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:18.437554 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03844b33-5234-49dc-bfe5-973485d15c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:18.456826 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-03844b33-5234-49dc-bfe5-973485d15c6e None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:18.458150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-03844b33-5234-49dc-bfe5-973485d15c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:18.467991 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1203/2411] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:18.472541 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06518204-8f9a-4f62-aa2f-bae2abd73a64 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:18.482554 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06518204-8f9a-4f62-aa2f-bae2abd73a64 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:18.487798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec55e3f-24de-40ca-a1ea-c63d590400ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:18.488469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec55e3f-24de-40ca-a1ea-c63d590400ab admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:18.488754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec55e3f-24de-40ca-a1ea-c63d590400ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:18.489099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec55e3f-24de-40ca-a1ea-c63d590400ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:18.489810 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec55e3f-24de-40ca-a1ea-c63d590400ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:46:18.490290 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec55e3f-24de-40ca-a1ea-c63d590400ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:18.491854 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec55e3f-24de-40ca-a1ea-c63d590400ab admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:18.504421 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1209/2412] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:46:18.509256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03844b33-5234-49dc-bfe5-973485d15c6e None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:18.519461 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03844b33-5234-49dc-bfe5-973485d15c6e None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:18.525151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93bd65-458f-4375-bef3-d2bb94eda7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:18.525830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93bd65-458f-4375-bef3-d2bb94eda7e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:18.527716 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93bd65-458f-4375-bef3-d2bb94eda7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:18.528332 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93bd65-458f-4375-bef3-d2bb94eda7e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:18.528845 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93bd65-458f-4375-bef3-d2bb94eda7e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:18.531070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93bd65-458f-4375-bef3-d2bb94eda7e9 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:18.565495 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1204/2413] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:46:18 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:46:18.566746 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec55e3f-24de-40ca-a1ea-c63d590400ab admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:18.576895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec55e3f-24de-40ca-a1ea-c63d590400ab admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:18.617567 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:18.645116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d83e0-2eca-4b04-929c-e733d408ef5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:18.646328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d83e0-2eca-4b04-929c-e733d408ef5e admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:18.646616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d83e0-2eca-4b04-929c-e733d408ef5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:18.646903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d83e0-2eca-4b04-929c-e733d408ef5e admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:18.647755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d83e0-2eca-4b04-929c-e733d408ef5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:18.650637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d83e0-2eca-4b04-929c-e733d408ef5e admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:18.680791 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1210/2414] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:46:18 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:46:18.682015 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93bd65-458f-4375-bef3-d2bb94eda7e9 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:18.692505 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93bd65-458f-4375-bef3-d2bb94eda7e9 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:18.697841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11ae19-d2cb-49d0-a84b-1052bbe05b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:18.698514 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11ae19-d2cb-49d0-a84b-1052bbe05b62 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:18.698820 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11ae19-d2cb-49d0-a84b-1052bbe05b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:18.699149 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11ae19-d2cb-49d0-a84b-1052bbe05b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:18.699883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11ae19-d2cb-49d0-a84b-1052bbe05b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:18.701680 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11ae19-d2cb-49d0-a84b-1052bbe05b62 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:18.712583 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1205/2415] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:46:18.716747 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d83e0-2eca-4b04-929c-e733d408ef5e admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:18.726816 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d83e0-2eca-4b04-929c-e733d408ef5e admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:18.731562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3f090-3138-4d94-b1f7-6de4e55cddbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:18.732219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3f090-3138-4d94-b1f7-6de4e55cddbc admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:18.732507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3f090-3138-4d94-b1f7-6de4e55cddbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:18.732787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3f090-3138-4d94-b1f7-6de4e55cddbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:18.733524 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3f090-3138-4d94-b1f7-6de4e55cddbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:18.735172 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c3f090-3138-4d94-b1f7-6de4e55cddbc admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:18.745316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1211/2416] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:46:18.749251 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11ae19-d2cb-49d0-a84b-1052bbe05b62 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:18.759329 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11ae19-d2cb-49d0-a84b-1052bbe05b62 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:18.764162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e6347-876a-42eb-b7a1-f95c9c531cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:18.765046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e6347-876a-42eb-b7a1-f95c9c531cc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:18.765331 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e6347-876a-42eb-b7a1-f95c9c531cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:18.765654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e6347-876a-42eb-b7a1-f95c9c531cc1 admin admin] PATH_INFO: `/v3/projects/cb4a02ef918f418f9509f0698503f428/users/5baede5b70674737ad285b5831e11002/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:18.781254 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e6347-876a-42eb-b7a1-f95c9c531cc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb4a02ef918f418f9509f0698503f428, user_id=5baede5b70674737ad285b5831e11002, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:18.783993 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e6347-876a-42eb-b7a1-f95c9c531cc1 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:18.793758 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:18.800088 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1206/2417] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:46:18 2022] PUT /identity/v3/projects/cb4a02ef918f418f9509f0698503f428/users/5baede5b70674737ad285b5831e11002/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:18.805394 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c6b2c-61d5-4e43-b5d4-2e97bcb03976 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:18.805667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c6b2c-61d5-4e43-b5d4-2e97bcb03976 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:18.805944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c6b2c-61d5-4e43-b5d4-2e97bcb03976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:18.826670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-a33c6b2c-61d5-4e43-b5d4-2e97bcb03976 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:18.828025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a33c6b2c-61d5-4e43-b5d4-2e97bcb03976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:18.892357 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1212/2418] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:18.902730 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e6347-876a-42eb-b7a1-f95c9c531cc1 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:18.904586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-559e6347-876a-42eb-b7a1-f95c9c531cc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:18.969630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e6347-876a-42eb-b7a1-f95c9c531cc1 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:18.973463 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a556e6-6546-4a5d-a1d5-76edc5c72160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:18.974134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a556e6-6546-4a5d-a1d5-76edc5c72160 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:18.974416 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a556e6-6546-4a5d-a1d5-76edc5c72160 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:18.974814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a556e6-6546-4a5d-a1d5-76edc5c72160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:18.985315 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a556e6-6546-4a5d-a1d5-76edc5c72160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:18.986840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a556e6-6546-4a5d-a1d5-76edc5c72160 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:18.998705 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1207/2419] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:46:19.131059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c6b2c-61d5-4e43-b5d4-2e97bcb03976 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:19.132567 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a33c6b2c-61d5-4e43-b5d4-2e97bcb03976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:19.145717 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c6b2c-61d5-4e43-b5d4-2e97bcb03976 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:19.151691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1c0c7-fda3-4cd4-8fa8-c89783c92a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:19.152604 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1c0c7-fda3-4cd4-8fa8-c89783c92a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:19.152931 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1c0c7-fda3-4cd4-8fa8-c89783c92a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:19.153162 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1c0c7-fda3-4cd4-8fa8-c89783c92a07 admin admin] PATH_INFO: `/v3/users/d1ccedf6fb8749ffb1773670c2041fe7` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:19.154752 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1c0c7-fda3-4cd4-8fa8-c89783c92a07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1ccedf6fb8749ffb1773670c2041fe7)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:19.156779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1c0c7-fda3-4cd4-8fa8-c89783c92a07 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:19.216977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-86b1c0c7-fda3-4cd4-8fa8-c89783c92a07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1ccedf6fb8749ffb1773670c2041fe7'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:19.232789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-86b1c0c7-fda3-4cd4-8fa8-c89783c92a07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1ccedf6fb8749ffb1773670c2041fe7'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:19.239855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-86b1c0c7-fda3-4cd4-8fa8-c89783c92a07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1ccedf6fb8749ffb1773670c2041fe7'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:19.240632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-86b1c0c7-fda3-4cd4-8fa8-c89783c92a07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1ccedf6fb8749ffb1773670c2041fe7'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:19.241449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-86b1c0c7-fda3-4cd4-8fa8-c89783c92a07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1ccedf6fb8749ffb1773670c2041fe7'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:19.257735 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1213/2420] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:46:19 2022] DELETE /identity/v3/users/d1ccedf6fb8749ffb1773670c2041fe7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:19.263570 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4d2bf-94cd-48d0-9e32-c50d186d7246 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:19.263926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4d2bf-94cd-48d0-9e32-c50d186d7246 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:19.264277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4d2bf-94cd-48d0-9e32-c50d186d7246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:19.287685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-07b4d2bf-94cd-48d0-9e32-c50d186d7246 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:19.288941 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-07b4d2bf-94cd-48d0-9e32-c50d186d7246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:19.364014 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1208/2421] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:19.382626 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1c0c7-fda3-4cd4-8fa8-c89783c92a07 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:19.384822 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-86b1c0c7-fda3-4cd4-8fa8-c89783c92a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:19.466112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:19.500880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1c0c7-fda3-4cd4-8fa8-c89783c92a07 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:19.505048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6c79f-beab-442d-bb76-ffc90e99b3f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:19.505720 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6c79f-beab-442d-bb76-ffc90e99b3f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:19.506003 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6c79f-beab-442d-bb76-ffc90e99b3f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:19.506277 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6c79f-beab-442d-bb76-ffc90e99b3f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:19.518248 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6c79f-beab-442d-bb76-ffc90e99b3f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:19.519789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6c79f-beab-442d-bb76-ffc90e99b3f5 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:19.531075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1214/2422] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:46:19.690399 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4d2bf-94cd-48d0-9e32-c50d186d7246 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:19.692437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-07b4d2bf-94cd-48d0-9e32-c50d186d7246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:19.706222 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4d2bf-94cd-48d0-9e32-c50d186d7246 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:19.711415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7160d-700c-455a-b278-c8a63e78e9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:19.712317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7160d-700c-455a-b278-c8a63e78e9eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:19.712602 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7160d-700c-455a-b278-c8a63e78e9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:19.713178 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7160d-700c-455a-b278-c8a63e78e9eb admin admin] PATH_INFO: `/v3/users/2bcae61f2e984751b6c68846fd718a50` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:19.714229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7160d-700c-455a-b278-c8a63e78e9eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bcae61f2e984751b6c68846fd718a50)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:19.716264 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7160d-700c-455a-b278-c8a63e78e9eb admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:19.767574 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b9f7160d-700c-455a-b278-c8a63e78e9eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bcae61f2e984751b6c68846fd718a50'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:19.783500 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b9f7160d-700c-455a-b278-c8a63e78e9eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bcae61f2e984751b6c68846fd718a50'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:19.796981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b9f7160d-700c-455a-b278-c8a63e78e9eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bcae61f2e984751b6c68846fd718a50'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:19.797926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b9f7160d-700c-455a-b278-c8a63e78e9eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bcae61f2e984751b6c68846fd718a50'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:19.798995 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b9f7160d-700c-455a-b278-c8a63e78e9eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bcae61f2e984751b6c68846fd718a50'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:19.825683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1209/2423] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:46:19 2022] DELETE /identity/v3/users/2bcae61f2e984751b6c68846fd718a50 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:19.832200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d691a-c2ba-4f8f-93e3-1ba4c3cd30f3 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:19.832499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d691a-c2ba-4f8f-93e3-1ba4c3cd30f3 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:19.832803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d691a-c2ba-4f8f-93e3-1ba4c3cd30f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:19.833083 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cd32a-2bb5-4d0a-88dc-71a2aa557c4c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:19.833559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cd32a-2bb5-4d0a-88dc-71a2aa557c4c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:19.834022 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479cd32a-2bb5-4d0a-88dc-71a2aa557c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:19.867942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-1c5d691a-c2ba-4f8f-93e3-1ba4c3cd30f3 None None] MFA Rules not processed for user `5baede5b70674737ad285b5831e11002`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:19.869103 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-479cd32a-2bb5-4d0a-88dc-71a2aa557c4c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:19.869232 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1c5d691a-c2ba-4f8f-93e3-1ba4c3cd30f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:19.870557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-479cd32a-2bb5-4d0a-88dc-71a2aa557c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:19.888749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:19.973086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1215/2424] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:19.974048 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d691a-c2ba-4f8f-93e3-1ba4c3cd30f3 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:19.976266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1c5d691a-c2ba-4f8f-93e3-1ba4c3cd30f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:19.984577 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1210/2425] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:19.994485 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d691a-c2ba-4f8f-93e3-1ba4c3cd30f3 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:19.994822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cd32a-2bb5-4d0a-88dc-71a2aa557c4c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:20.002021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-479cd32a-2bb5-4d0a-88dc-71a2aa557c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:20.002253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:20.003727 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:20.004078 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:20.004431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0 admin admin] PATH_INFO: `/v3/projects/92141c036fca417cbc3f4a7ebad30700` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:20.006021 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92141c036fca417cbc3f4a7ebad30700)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:20.008196 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:20.071542 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92141c036fca417cbc3f4a7ebad30700', 'request_id': 'req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:20.072098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479cd32a-2bb5-4d0a-88dc-71a2aa557c4c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:20.076925 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92141c036fca417cbc3f4a7ebad30700', 'request_id': 'req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:20.090392 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92141c036fca417cbc3f4a7ebad30700', 'request_id': 'req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:20.102429 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0 admin admin] The token cache is being invalidate because project 92141c036fca417cbc3f4a7ebad30700 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:46:20.102835 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:20.103587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:20.105238 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1216/2426] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:46:19 2022] DELETE /identity/v3/projects/92141c036fca417cbc3f4a7ebad30700 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:20.105996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:20.107287 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:20.123666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ba642-7a74-4233-a12f-a608a860a4da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:20.124334 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ba642-7a74-4233-a12f-a608a860a4da service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:20.124616 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ba642-7a74-4233-a12f-a608a860a4da service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:20.124890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ba642-7a74-4233-a12f-a608a860a4da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:20.126730 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-891ba642-7a74-4233-a12f-a608a860a4da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:20.169537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:20.182341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3cde7-4f75-4d35-96ac-1fcdd66790e0 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:20.187212 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739205ce-2485-4ef3-aebe-c3662da049b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:20.188115 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739205ce-2485-4ef3-aebe-c3662da049b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:20.188544 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739205ce-2485-4ef3-aebe-c3662da049b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:20.188990 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739205ce-2485-4ef3-aebe-c3662da049b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:20.191031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-739205ce-2485-4ef3-aebe-c3662da049b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:20.202639 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ba642-7a74-4233-a12f-a608a860a4da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:20.205937 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ba642-7a74-4233-a12f-a608a860a4da service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:20.222900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1211/2427] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:46:20.224837 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141ce63-895e-4cf4-994c-2620c343d0e8 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:20.231410 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141ce63-895e-4cf4-994c-2620c343d0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:20.231697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141ce63-895e-4cf4-994c-2620c343d0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:20.250839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-9141ce63-895e-4cf4-994c-2620c343d0e8 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:20.252201 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9141ce63-895e-4cf4-994c-2620c343d0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:20.267049 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1212/2428] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:20.273963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14613cda-f4cc-4acf-baa2-df0ef450f413 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:20.274255 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14613cda-f4cc-4acf-baa2-df0ef450f413 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:20.274539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14613cda-f4cc-4acf-baa2-df0ef450f413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:20.274725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739205ce-2485-4ef3-aebe-c3662da049b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:20.276490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739205ce-2485-4ef3-aebe-c3662da049b0 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:20.291057 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1217/2429] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:46:20.305215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-14613cda-f4cc-4acf-baa2-df0ef450f413 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:20.306660 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-14613cda-f4cc-4acf-baa2-df0ef450f413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:20.322907 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1213/2430] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:46:20.323697 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739205ce-2485-4ef3-aebe-c3662da049b0 service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:20.348467 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739205ce-2485-4ef3-aebe-c3662da049b0 service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:20.354985 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06189c-7dab-46cc-ae6f-8b1da902d0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:20.355881 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06189c-7dab-46cc-ae6f-8b1da902d0cc admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:20.356362 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06189c-7dab-46cc-ae6f-8b1da902d0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:20.356648 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06189c-7dab-46cc-ae6f-8b1da902d0cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:20.357521 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06189c-7dab-46cc-ae6f-8b1da902d0cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:46:20.357798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06189c-7dab-46cc-ae6f-8b1da902d0cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:20.359693 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06189c-7dab-46cc-ae6f-8b1da902d0cc admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:20.375852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1218/2431] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:46:20.380489 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14613cda-f4cc-4acf-baa2-df0ef450f413 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:20.390700 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14613cda-f4cc-4acf-baa2-df0ef450f413 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:20.408527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b82aa-f635-4077-90ad-ccd2de35d134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:20.409593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b82aa-f635-4077-90ad-ccd2de35d134 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:20.409875 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b82aa-f635-4077-90ad-ccd2de35d134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:20.410146 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b82aa-f635-4077-90ad-ccd2de35d134 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:20.410921 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b82aa-f635-4077-90ad-ccd2de35d134 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:20.412912 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b82aa-f635-4077-90ad-ccd2de35d134 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:20.416585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:20.466882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1214/2432] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:46:20 2022] POST /identity/v3/projects => generated 375 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:46:20.471937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06189c-7dab-46cc-ae6f-8b1da902d0cc admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:20.496802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06189c-7dab-46cc-ae6f-8b1da902d0cc admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:20.620446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b82aa-f635-4077-90ad-ccd2de35d134 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:20.622578 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-c10b82aa-f635-4077-90ad-ccd2de35d134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:20.632347 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f4daa-42ef-43d5-a604-57c8594577be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:20.637417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f4daa-42ef-43d5-a604-57c8594577be admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:20.637783 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f4daa-42ef-43d5-a604-57c8594577be admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:20.638261 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f4daa-42ef-43d5-a604-57c8594577be admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:20.639416 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f4daa-42ef-43d5-a604-57c8594577be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:20.641477 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f4daa-42ef-43d5-a604-57c8594577be admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:20.656403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b82aa-f635-4077-90ad-ccd2de35d134 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:20.673191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d694b-2022-4793-a7b3-4c479a8fcddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:20.674166 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d694b-2022-4793-a7b3-4c479a8fcddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:20.674504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d694b-2022-4793-a7b3-4c479a8fcddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:20.674788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d694b-2022-4793-a7b3-4c479a8fcddf admin admin] PATH_INFO: `/v3/projects/abe1c0449c034c2484a9217a7ef93fea` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:20.676158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d694b-2022-4793-a7b3-4c479a8fcddf admin admin] RBAC: Authorizing `identity:delete_project(project_id=abe1c0449c034c2484a9217a7ef93fea)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:20.678603 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d694b-2022-4793-a7b3-4c479a8fcddf admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:20.699357 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1219/2433] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:46:20 2022] POST /identity/v3/users => generated 288 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:46:20.700441 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f4daa-42ef-43d5-a604-57c8594577be admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:20.713350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f4daa-42ef-43d5-a604-57c8594577be admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:20.724648 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81b937-cc03-4810-a9fd-837a4bc3c20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:20.725812 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81b937-cc03-4810-a9fd-837a4bc3c20b admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:20.726239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81b937-cc03-4810-a9fd-837a4bc3c20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:20.726587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81b937-cc03-4810-a9fd-837a4bc3c20b admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:20.727494 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81b937-cc03-4810-a9fd-837a4bc3c20b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:20.729653 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81b937-cc03-4810-a9fd-837a4bc3c20b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:20.749226 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1220/2434] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:46:20.756652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81b937-cc03-4810-a9fd-837a4bc3c20b admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:20.769460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81b937-cc03-4810-a9fd-837a4bc3c20b admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:20.773163 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-108d694b-2022-4793-a7b3-4c479a8fcddf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abe1c0449c034c2484a9217a7ef93fea', 'request_id': 'req-108d694b-2022-4793-a7b3-4c479a8fcddf'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:20.782635 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-108d694b-2022-4793-a7b3-4c479a8fcddf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abe1c0449c034c2484a9217a7ef93fea', 'request_id': 'req-108d694b-2022-4793-a7b3-4c479a8fcddf'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:20.790953 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-108d694b-2022-4793-a7b3-4c479a8fcddf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abe1c0449c034c2484a9217a7ef93fea', 'request_id': 'req-108d694b-2022-4793-a7b3-4c479a8fcddf'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:20.804693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-108d694b-2022-4793-a7b3-4c479a8fcddf admin admin] The token cache is being invalidate because project abe1c0449c034c2484a9217a7ef93fea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:46:20.804979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-108d694b-2022-4793-a7b3-4c479a8fcddf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:20.806087 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-108d694b-2022-4793-a7b3-4c479a8fcddf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:20.810796 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1215/2435] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:46:20 2022] DELETE /identity/v3/projects/abe1c0449c034c2484a9217a7ef93fea => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:20.813166 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d694b-2022-4793-a7b3-4c479a8fcddf admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:20.814811 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-108d694b-2022-4793-a7b3-4c479a8fcddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:20.850583 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc695c-21c3-4f69-a498-5cb1870bdf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:20.851316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc695c-21c3-4f69-a498-5cb1870bdf67 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:20.851603 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc695c-21c3-4f69-a498-5cb1870bdf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:20.852017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc695c-21c3-4f69-a498-5cb1870bdf67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:20.852751 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc695c-21c3-4f69-a498-5cb1870bdf67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:20.854810 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc695c-21c3-4f69-a498-5cb1870bdf67 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:20.862376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:20.872024 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1221/2436] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:20 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:46:20.877637 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc695c-21c3-4f69-a498-5cb1870bdf67 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:20.880514 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-86dc695c-21c3-4f69-a498-5cb1870bdf67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:20.897109 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc695c-21c3-4f69-a498-5cb1870bdf67 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:20.902893 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db913bce-ab6f-4e40-b507-15f65626ec89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:20.904374 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db913bce-ab6f-4e40-b507-15f65626ec89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:20.904736 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db913bce-ab6f-4e40-b507-15f65626ec89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:20.905202 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db913bce-ab6f-4e40-b507-15f65626ec89 admin admin] PATH_INFO: `/v3/projects/72acefad68e9438e8c025c229ac36607/users/2288b91f37ea46adb453edbb274058e4/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:20.906442 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d694b-2022-4793-a7b3-4c479a8fcddf admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:20.913659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ae958-b5e2-46e6-88d1-42036540dbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:20.914914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ae958-b5e2-46e6-88d1-42036540dbb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:20.915378 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ae958-b5e2-46e6-88d1-42036540dbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:20.917763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ae958-b5e2-46e6-88d1-42036540dbb0 admin admin] PATH_INFO: `/v3/users/0e250375bd6944e99c20e76676023d69` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:20.919773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ae958-b5e2-46e6-88d1-42036540dbb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e250375bd6944e99c20e76676023d69)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:20.922588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ae958-b5e2-46e6-88d1-42036540dbb0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:20.927260 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db913bce-ab6f-4e40-b507-15f65626ec89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72acefad68e9438e8c025c229ac36607, user_id=2288b91f37ea46adb453edbb274058e4, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:20.931818 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db913bce-ab6f-4e40-b507-15f65626ec89 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:20.952320 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1222/2437] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:46:20 2022] PUT /identity/v3/projects/72acefad68e9438e8c025c229ac36607/users/2288b91f37ea46adb453edbb274058e4/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:20.965250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad938704-3ea0-4d1c-a997-6ebae16e39b2 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:20.965398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad938704-3ea0-4d1c-a997-6ebae16e39b2 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:20.965656 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad938704-3ea0-4d1c-a997-6ebae16e39b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:20.982454 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-fc2ae958-b5e2-46e6-88d1-42036540dbb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e250375bd6944e99c20e76676023d69'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:20.998302 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-ad938704-3ea0-4d1c-a997-6ebae16e39b2 None None] MFA Rules not processed for user `2288b91f37ea46adb453edbb274058e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:20.999708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ad938704-3ea0-4d1c-a997-6ebae16e39b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:21.000947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-fc2ae958-b5e2-46e6-88d1-42036540dbb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e250375bd6944e99c20e76676023d69'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:21.006248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=82,keystone:INSERT=2,keystone:DELETE=16 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:21.010152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-fc2ae958-b5e2-46e6-88d1-42036540dbb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e250375bd6944e99c20e76676023d69'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:21.011473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-fc2ae958-b5e2-46e6-88d1-42036540dbb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e250375bd6944e99c20e76676023d69'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:21.012213 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-fc2ae958-b5e2-46e6-88d1-42036540dbb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e250375bd6944e99c20e76676023d69'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:21.028264 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1216/2438] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:46:20 2022] DELETE /identity/v3/users/0e250375bd6944e99c20e76676023d69 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:21.086969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1223/2439] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:46:21.099284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ae958-b5e2-46e6-88d1-42036540dbb0 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:21.100772 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-fc2ae958-b5e2-46e6-88d1-42036540dbb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:21.174948 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ae958-b5e2-46e6-88d1-42036540dbb0 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:21.178676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c90173-d0ba-4f53-be0c-ab6457542516 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:21.179453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c90173-d0ba-4f53-be0c-ab6457542516 service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:21.179658 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c90173-d0ba-4f53-be0c-ab6457542516 service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:21.179934 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c90173-d0ba-4f53-be0c-ab6457542516 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:21.242836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c90173-d0ba-4f53-be0c-ab6457542516 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:21.244392 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c90173-d0ba-4f53-be0c-ab6457542516 service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:21.253389 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1217/2440] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:46:21.320152 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad938704-3ea0-4d1c-a997-6ebae16e39b2 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:21.321643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ad938704-3ea0-4d1c-a997-6ebae16e39b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:21.395692 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad938704-3ea0-4d1c-a997-6ebae16e39b2 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:21.463222 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752cc78-4fb5-495c-ad41-f499be208344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:21.464108 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752cc78-4fb5-495c-ad41-f499be208344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:21.464394 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752cc78-4fb5-495c-ad41-f499be208344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:21.464668 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752cc78-4fb5-495c-ad41-f499be208344 admin admin] PATH_INFO: `/v3/projects/b10643783e274469a9c5113dfbb0ba14` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:21.465945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752cc78-4fb5-495c-ad41-f499be208344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10643783e274469a9c5113dfbb0ba14)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:21.467872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752cc78-4fb5-495c-ad41-f499be208344 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:21.498626 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:21.560377 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-d752cc78-4fb5-495c-ad41-f499be208344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10643783e274469a9c5113dfbb0ba14', 'request_id': 'req-d752cc78-4fb5-495c-ad41-f499be208344'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:21.576108 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-d752cc78-4fb5-495c-ad41-f499be208344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10643783e274469a9c5113dfbb0ba14', 'request_id': 'req-d752cc78-4fb5-495c-ad41-f499be208344'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:21.604075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-d752cc78-4fb5-495c-ad41-f499be208344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10643783e274469a9c5113dfbb0ba14', 'request_id': 'req-d752cc78-4fb5-495c-ad41-f499be208344'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:21.628396 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-d752cc78-4fb5-495c-ad41-f499be208344 admin admin] The token cache is being invalidate because project b10643783e274469a9c5113dfbb0ba14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:46:21.636615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-d752cc78-4fb5-495c-ad41-f499be208344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:21.638991 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-d752cc78-4fb5-495c-ad41-f499be208344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:21.646343 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1224/2441] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:46:21 2022] DELETE /identity/v3/projects/b10643783e274469a9c5113dfbb0ba14 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:21.653334 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fb8da-f3e2-43d1-a9af-75246fe43091 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:21.653633 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fb8da-f3e2-43d1-a9af-75246fe43091 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:21.653911 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fb8da-f3e2-43d1-a9af-75246fe43091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:21.680750 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-f80fb8da-f3e2-43d1-a9af-75246fe43091 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:21.682315 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f80fb8da-f3e2-43d1-a9af-75246fe43091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:21.765636 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1218/2442] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:21.774598 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73f904-7e2a-47af-baa2-46338fa58f27 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:21.775197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73f904-7e2a-47af-baa2-46338fa58f27 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:21.775411 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73f904-7e2a-47af-baa2-46338fa58f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:21.799371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-1f73f904-7e2a-47af-baa2-46338fa58f27 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:21.800615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1f73f904-7e2a-47af-baa2-46338fa58f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:21.808751 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1225/2443] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:46:21.813058 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80fb8da-f3e2-43d1-a9af-75246fe43091 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:21.821059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3691c-4693-4e68-9db6-16a4cde19719 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:21.821350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3691c-4693-4e68-9db6-16a4cde19719 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:21.821694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3691c-4693-4e68-9db6-16a4cde19719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:21.824029 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80fb8da-f3e2-43d1-a9af-75246fe43091 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:21.828666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2129226-c34d-46c6-bab8-4b59f619c2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:21.829329 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2129226-c34d-46c6-bab8-4b59f619c2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:21.829613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2129226-c34d-46c6-bab8-4b59f619c2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:21.829895 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2129226-c34d-46c6-bab8-4b59f619c2a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:21.830626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2129226-c34d-46c6-bab8-4b59f619c2a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:46:21.830935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2129226-c34d-46c6-bab8-4b59f619c2a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:21.832579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2129226-c34d-46c6-bab8-4b59f619c2a2 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:21.839721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-bad3691c-4693-4e68-9db6-16a4cde19719 None None] MFA Rules not processed for user `5baede5b70674737ad285b5831e11002`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:21.840929 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-bad3691c-4693-4e68-9db6-16a4cde19719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:21.844393 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1219/2444] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:46:21.848876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2129226-c34d-46c6-bab8-4b59f619c2a2 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:21.863050 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2129226-c34d-46c6-bab8-4b59f619c2a2 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:21.867978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2cc28-ea27-4abb-9ad8-9e83e57a606b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:21.868740 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2cc28-ea27-4abb-9ad8-9e83e57a606b admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:21.869043 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2cc28-ea27-4abb-9ad8-9e83e57a606b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:21.869318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2cc28-ea27-4abb-9ad8-9e83e57a606b admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:21.869989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2cc28-ea27-4abb-9ad8-9e83e57a606b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:21.871992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2cc28-ea27-4abb-9ad8-9e83e57a606b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:21.898469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1226/2445] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:46:21.898992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1220/2446] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:46:21 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:46:21.902995 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3691c-4693-4e68-9db6-16a4cde19719 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:21.914559 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3691c-4693-4e68-9db6-16a4cde19719 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:21.976363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73e36a-5f68-422a-9017-f5135ea22389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:21.977053 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73e36a-5f68-422a-9017-f5135ea22389 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:21.977338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73e36a-5f68-422a-9017-f5135ea22389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:21.977645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a73e36a-5f68-422a-9017-f5135ea22389 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:21.978341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a73e36a-5f68-422a-9017-f5135ea22389 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:21.980396 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a73e36a-5f68-422a-9017-f5135ea22389 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:22.001125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1227/2447] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:46:21 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:46:22.005315 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2cc28-ea27-4abb-9ad8-9e83e57a606b admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:22.015634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2cc28-ea27-4abb-9ad8-9e83e57a606b admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:22.020736 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9201e06-1cf1-4cc9-a529-ff22d814c0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:22.021429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9201e06-1cf1-4cc9-a529-ff22d814c0ec admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:22.021697 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9201e06-1cf1-4cc9-a529-ff22d814c0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:22.021995 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9201e06-1cf1-4cc9-a529-ff22d814c0ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:22.022793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9201e06-1cf1-4cc9-a529-ff22d814c0ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:22.024458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9201e06-1cf1-4cc9-a529-ff22d814c0ec admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:22.026068 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:22.036914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1221/2448] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:46:22.040713 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73e36a-5f68-422a-9017-f5135ea22389 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:22.050724 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a73e36a-5f68-422a-9017-f5135ea22389 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:22.055758 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7f0fa-3054-4770-8e17-26200f806118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:22.056435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7f0fa-3054-4770-8e17-26200f806118 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:22.056718 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7f0fa-3054-4770-8e17-26200f806118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:22.057042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7f0fa-3054-4770-8e17-26200f806118 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:22.057772 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7f0fa-3054-4770-8e17-26200f806118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:22.059437 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7f0fa-3054-4770-8e17-26200f806118 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:22.070087 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1228/2449] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:46:22.074262 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9201e06-1cf1-4cc9-a529-ff22d814c0ec admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:22.084445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9201e06-1cf1-4cc9-a529-ff22d814c0ec admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:22.089794 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013e86b-59bc-4f43-849d-ce2340633d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:22.090711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e86b-59bc-4f43-849d-ce2340633d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:22.090995 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e86b-59bc-4f43-849d-ce2340633d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:22.091310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013e86b-59bc-4f43-849d-ce2340633d77 admin admin] PATH_INFO: `/v3/projects/e23fb259ba2b457a9eae329042f6fddd/users/c64d05b8e361432bb0ad5c2b5e162e85/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:22.106739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013e86b-59bc-4f43-849d-ce2340633d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e23fb259ba2b457a9eae329042f6fddd, user_id=c64d05b8e361432bb0ad5c2b5e162e85, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:22.109383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013e86b-59bc-4f43-849d-ce2340633d77 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:22.133433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1222/2450] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:46:22 2022] PUT /identity/v3/projects/e23fb259ba2b457a9eae329042f6fddd/users/c64d05b8e361432bb0ad5c2b5e162e85/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:22.135122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f15b6d-7445-4dcd-8ba9-46bc1ed5a987 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:22.135488 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f15b6d-7445-4dcd-8ba9-46bc1ed5a987 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:22.135847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f15b6d-7445-4dcd-8ba9-46bc1ed5a987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:22.163252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-b9f15b6d-7445-4dcd-8ba9-46bc1ed5a987 None None] MFA Rules not processed for user `c64d05b8e361432bb0ad5c2b5e162e85`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:22.164566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b9f15b6d-7445-4dcd-8ba9-46bc1ed5a987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:22.173530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:22.233319 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1229/2451] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:46:22.247098 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013e86b-59bc-4f43-849d-ce2340633d77 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:22.248964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7013e86b-59bc-4f43-849d-ce2340633d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:22.310062 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013e86b-59bc-4f43-849d-ce2340633d77 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:22.314515 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b0858-4fff-4afa-af3e-ddb1f6444c11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:22.315207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b0858-4fff-4afa-af3e-ddb1f6444c11 service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:22.315495 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b0858-4fff-4afa-af3e-ddb1f6444c11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:22.315768 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b0858-4fff-4afa-af3e-ddb1f6444c11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:22.325944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b0858-4fff-4afa-af3e-ddb1f6444c11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:22.329214 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b0858-4fff-4afa-af3e-ddb1f6444c11 service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:22.338794 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1223/2452] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:46:22.358544 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f15b6d-7445-4dcd-8ba9-46bc1ed5a987 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:22.429677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f15b6d-7445-4dcd-8ba9-46bc1ed5a987 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:22.434361 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb32cd-ad56-4153-9ec4-0c677926900f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:22.435549 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb32cd-ad56-4153-9ec4-0c677926900f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:22.435924 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb32cd-ad56-4153-9ec4-0c677926900f admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:22.436212 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb32cd-ad56-4153-9ec4-0c677926900f admin admin] PATH_INFO: `/v3/users/c64d05b8e361432bb0ad5c2b5e162e85` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:22.437468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb32cd-ad56-4153-9ec4-0c677926900f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c64d05b8e361432bb0ad5c2b5e162e85)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:22.439503 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb32cd-ad56-4153-9ec4-0c677926900f admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:22.499706 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-eddb32cd-ad56-4153-9ec4-0c677926900f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c64d05b8e361432bb0ad5c2b5e162e85'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:22.516279 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-eddb32cd-ad56-4153-9ec4-0c677926900f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c64d05b8e361432bb0ad5c2b5e162e85'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:22.523595 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-eddb32cd-ad56-4153-9ec4-0c677926900f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c64d05b8e361432bb0ad5c2b5e162e85'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:22.524418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-eddb32cd-ad56-4153-9ec4-0c677926900f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c64d05b8e361432bb0ad5c2b5e162e85'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:22.525035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-eddb32cd-ad56-4153-9ec4-0c677926900f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c64d05b8e361432bb0ad5c2b5e162e85'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:22.540161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1230/2453] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:46:22 2022] DELETE /identity/v3/users/c64d05b8e361432bb0ad5c2b5e162e85 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:22.544739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63afea0-ea96-4f52-a615-cd84e8659c12 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:22.545060 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63afea0-ea96-4f52-a615-cd84e8659c12 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:22.545392 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63afea0-ea96-4f52-a615-cd84e8659c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:22.562979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-f63afea0-ea96-4f52-a615-cd84e8659c12 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:22.564268 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f63afea0-ea96-4f52-a615-cd84e8659c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:22.629263 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1224/2454] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:22.641555 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb32cd-ad56-4153-9ec4-0c677926900f admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:22.643052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-eddb32cd-ad56-4153-9ec4-0c677926900f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:22.705325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:22.755082 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb32cd-ad56-4153-9ec4-0c677926900f admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:22.759261 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1023ba-e9d7-404a-8883-be0b449befae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:22.760037 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1023ba-e9d7-404a-8883-be0b449befae service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:22.760509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1023ba-e9d7-404a-8883-be0b449befae service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:22.760900 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1023ba-e9d7-404a-8883-be0b449befae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:22.793027 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1023ba-e9d7-404a-8883-be0b449befae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:22.794513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1023ba-e9d7-404a-8883-be0b449befae service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:22.807882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1231/2455] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:46:23.218691 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63afea0-ea96-4f52-a615-cd84e8659c12 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:23.220192 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f63afea0-ea96-4f52-a615-cd84e8659c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:23.237674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63afea0-ea96-4f52-a615-cd84e8659c12 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:23.243604 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665a2bc-8ee7-46a6-a094-d427b909a9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:23.244506 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665a2bc-8ee7-46a6-a094-d427b909a9e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:23.244797 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665a2bc-8ee7-46a6-a094-d427b909a9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:23.245070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665a2bc-8ee7-46a6-a094-d427b909a9e2 admin admin] PATH_INFO: `/v3/projects/e23fb259ba2b457a9eae329042f6fddd` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:23.246518 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665a2bc-8ee7-46a6-a094-d427b909a9e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e23fb259ba2b457a9eae329042f6fddd)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:23.248588 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665a2bc-8ee7-46a6-a094-d427b909a9e2 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:23.310374 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-2665a2bc-8ee7-46a6-a094-d427b909a9e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e23fb259ba2b457a9eae329042f6fddd', 'request_id': 'req-2665a2bc-8ee7-46a6-a094-d427b909a9e2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:23.318902 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-2665a2bc-8ee7-46a6-a094-d427b909a9e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e23fb259ba2b457a9eae329042f6fddd', 'request_id': 'req-2665a2bc-8ee7-46a6-a094-d427b909a9e2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:23.326494 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-2665a2bc-8ee7-46a6-a094-d427b909a9e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e23fb259ba2b457a9eae329042f6fddd', 'request_id': 'req-2665a2bc-8ee7-46a6-a094-d427b909a9e2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:23.336232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:23.342035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-2665a2bc-8ee7-46a6-a094-d427b909a9e2 admin admin] The token cache is being invalidate because project e23fb259ba2b457a9eae329042f6fddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:46:23.342555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-2665a2bc-8ee7-46a6-a094-d427b909a9e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:23.343366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-2665a2bc-8ee7-46a6-a094-d427b909a9e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:23.345617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1225/2456] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:46:23 2022] DELETE /identity/v3/projects/e23fb259ba2b457a9eae329042f6fddd => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:23.350441 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a434d4-2b7c-4b8d-971a-57a1814e3817 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:23.350796 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a434d4-2b7c-4b8d-971a-57a1814e3817 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:23.351112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a434d4-2b7c-4b8d-971a-57a1814e3817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:23.369636 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-b7a434d4-2b7c-4b8d-971a-57a1814e3817 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:23.370858 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b7a434d4-2b7c-4b8d-971a-57a1814e3817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:23.434273 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1232/2457] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:23.446987 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6af3c-67c0-412e-aff7-6a0adde6258c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:23.447241 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6af3c-67c0-412e-aff7-6a0adde6258c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:23.447520 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6af3c-67c0-412e-aff7-6a0adde6258c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:23.468745 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-afd6af3c-67c0-412e-aff7-6a0adde6258c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:23.469963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-afd6af3c-67c0-412e-aff7-6a0adde6258c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:23.480124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1226/2458] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:23.484482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a434d4-2b7c-4b8d-971a-57a1814e3817 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:23.494730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a434d4-2b7c-4b8d-971a-57a1814e3817 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:23.499750 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33806e05-168e-442e-8e0a-cb6b760923b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:23.500465 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33806e05-168e-442e-8e0a-cb6b760923b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:23.500738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33806e05-168e-442e-8e0a-cb6b760923b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:23.501031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33806e05-168e-442e-8e0a-cb6b760923b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:23.501762 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33806e05-168e-442e-8e0a-cb6b760923b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:46:23.502070 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33806e05-168e-442e-8e0a-cb6b760923b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:23.503732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33806e05-168e-442e-8e0a-cb6b760923b0 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:23.515816 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1233/2459] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:46:23.520247 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6af3c-67c0-412e-aff7-6a0adde6258c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:23.531965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6af3c-67c0-412e-aff7-6a0adde6258c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:23.538127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03c0ec-526a-48ac-9caf-a14ac100e9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:23.538801 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03c0ec-526a-48ac-9caf-a14ac100e9cb admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:23.539120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03c0ec-526a-48ac-9caf-a14ac100e9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:23.539396 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03c0ec-526a-48ac-9caf-a14ac100e9cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:23.540096 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae03c0ec-526a-48ac-9caf-a14ac100e9cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:23.542021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae03c0ec-526a-48ac-9caf-a14ac100e9cb admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:23.586946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1227/2460] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:46:23 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:46:23.591662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33806e05-168e-442e-8e0a-cb6b760923b0 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:23.605347 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33806e05-168e-442e-8e0a-cb6b760923b0 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:23.671825 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37d13d-4a42-4e16-8bfb-d2020f545e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:23.672517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37d13d-4a42-4e16-8bfb-d2020f545e25 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:23.672805 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37d13d-4a42-4e16-8bfb-d2020f545e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:23.673798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37d13d-4a42-4e16-8bfb-d2020f545e25 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:23.674866 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37d13d-4a42-4e16-8bfb-d2020f545e25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:23.676936 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e37d13d-4a42-4e16-8bfb-d2020f545e25 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:23.701523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1234/2461] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:46:23 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:46:23.705387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03c0ec-526a-48ac-9caf-a14ac100e9cb admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:23.719146 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03c0ec-526a-48ac-9caf-a14ac100e9cb admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:23.728347 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d1c78-f089-4e06-8560-399af540ae22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:23.729043 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d1c78-f089-4e06-8560-399af540ae22 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:23.729323 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d1c78-f089-4e06-8560-399af540ae22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:23.729592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d1c78-f089-4e06-8560-399af540ae22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:23.730321 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d1c78-f089-4e06-8560-399af540ae22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:23.732092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d1c78-f089-4e06-8560-399af540ae22 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:23.743398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1228/2462] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:46:23.747846 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37d13d-4a42-4e16-8bfb-d2020f545e25 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:23.766484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37d13d-4a42-4e16-8bfb-d2020f545e25 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:23.773954 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d22b1-09e4-49e3-99bd-a0c9ad955915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:23.774641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d22b1-09e4-49e3-99bd-a0c9ad955915 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:23.774925 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d22b1-09e4-49e3-99bd-a0c9ad955915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:23.775338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d22b1-09e4-49e3-99bd-a0c9ad955915 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:23.776182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d22b1-09e4-49e3-99bd-a0c9ad955915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:23.777883 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d22b1-09e4-49e3-99bd-a0c9ad955915 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:23.789141 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1235/2463] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:46:23.793426 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d1c78-f089-4e06-8560-399af540ae22 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:23.803582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d1c78-f089-4e06-8560-399af540ae22 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:23.814897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fff8a-2af8-4337-bafb-42daafbb1a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:23.815837 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fff8a-2af8-4337-bafb-42daafbb1a94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:23.816230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fff8a-2af8-4337-bafb-42daafbb1a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:23.816430 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fff8a-2af8-4337-bafb-42daafbb1a94 admin admin] PATH_INFO: `/v3/projects/0f555c1188454dd2a5132e9c80a1e262/users/2efb609ec03343c8be1f4cdb977c48c8/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:23.832679 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9fff8a-2af8-4337-bafb-42daafbb1a94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f555c1188454dd2a5132e9c80a1e262, user_id=2efb609ec03343c8be1f4cdb977c48c8, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:23.835572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9fff8a-2af8-4337-bafb-42daafbb1a94 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:23.849684 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1229/2464] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:46:23 2022] PUT /identity/v3/projects/0f555c1188454dd2a5132e9c80a1e262/users/2efb609ec03343c8be1f4cdb977c48c8/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:23.860551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd9b70-aacd-4362-b220-88bcfcbed59c None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:23.860847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd9b70-aacd-4362-b220-88bcfcbed59c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:23.861143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd9b70-aacd-4362-b220-88bcfcbed59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:23.893981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-e9fd9b70-aacd-4362-b220-88bcfcbed59c None None] MFA Rules not processed for user `2efb609ec03343c8be1f4cdb977c48c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:23.895255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e9fd9b70-aacd-4362-b220-88bcfcbed59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:23.922538 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:23.967630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1236/2465] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:23.978873 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fff8a-2af8-4337-bafb-42daafbb1a94 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:23.980659 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ca9fff8a-2af8-4337-bafb-42daafbb1a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:24.037550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fff8a-2af8-4337-bafb-42daafbb1a94 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:24.041533 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e2f43-352e-4bd3-a66c-49d09ed7e9eb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2707c6967a0a419a98ae142ff1d1a961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:24.042189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e2f43-352e-4bd3-a66c-49d09ed7e9eb service glance] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:24.042532 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e2f43-352e-4bd3-a66c-49d09ed7e9eb service glance] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:24.042879 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e2f43-352e-4bd3-a66c-49d09ed7e9eb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:24.052054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e2f43-352e-4bd3-a66c-49d09ed7e9eb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:24.053544 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e2f43-352e-4bd3-a66c-49d09ed7e9eb service glance] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:24.062684 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1230/2466] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:46:33.002445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd9b70-aacd-4362-b220-88bcfcbed59c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:33.004085 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e9fd9b70-aacd-4362-b220-88bcfcbed59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:33.073831 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd9b70-aacd-4362-b220-88bcfcbed59c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:33.078877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8fd26-ee82-464a-a6cd-3c5eebd29ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:33.079825 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8fd26-ee82-464a-a6cd-3c5eebd29ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:33.080104 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8fd26-ee82-464a-a6cd-3c5eebd29ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:33.080376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8fd26-ee82-464a-a6cd-3c5eebd29ea5 admin admin] PATH_INFO: `/v3/users/2288b91f37ea46adb453edbb274058e4` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:33.081582 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8fd26-ee82-464a-a6cd-3c5eebd29ea5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2288b91f37ea46adb453edbb274058e4)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:33.083562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8fd26-ee82-464a-a6cd-3c5eebd29ea5 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:33.136487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5bb8fd26-ee82-464a-a6cd-3c5eebd29ea5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2288b91f37ea46adb453edbb274058e4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:33.155225 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5bb8fd26-ee82-464a-a6cd-3c5eebd29ea5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2288b91f37ea46adb453edbb274058e4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:33.162624 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5bb8fd26-ee82-464a-a6cd-3c5eebd29ea5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2288b91f37ea46adb453edbb274058e4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:33.163730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5bb8fd26-ee82-464a-a6cd-3c5eebd29ea5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2288b91f37ea46adb453edbb274058e4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:33.164708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5bb8fd26-ee82-464a-a6cd-3c5eebd29ea5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2288b91f37ea46adb453edbb274058e4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:33.188080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1237/2467] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:46:33 2022] DELETE /identity/v3/users/2288b91f37ea46adb453edbb274058e4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:33.192498 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e6854-9d52-4e47-ad51-51e973962aa1 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:33.193020 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e6854-9d52-4e47-ad51-51e973962aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:33.193124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e6854-9d52-4e47-ad51-51e973962aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:33.214802 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-aa2e6854-9d52-4e47-ad51-51e973962aa1 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:33.216343 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-aa2e6854-9d52-4e47-ad51-51e973962aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:33.318830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1231/2468] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:33.332774 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8fd26-ee82-464a-a6cd-3c5eebd29ea5 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:33.334810 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5bb8fd26-ee82-464a-a6cd-3c5eebd29ea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:33.394362 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:33.400264 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8fd26-ee82-464a-a6cd-3c5eebd29ea5 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:33.405125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a352f-efa0-4d35-8405-07210555f748 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:33.406094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a352f-efa0-4d35-8405-07210555f748 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:33.406478 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a352f-efa0-4d35-8405-07210555f748 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:33.406980 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a352f-efa0-4d35-8405-07210555f748 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:33.422705 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a352f-efa0-4d35-8405-07210555f748 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:33.424280 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a352f-efa0-4d35-8405-07210555f748 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:33.434609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1238/2469] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:46:33.724939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e6854-9d52-4e47-ad51-51e973962aa1 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:33.726457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-aa2e6854-9d52-4e47-ad51-51e973962aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:33.740715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e6854-9d52-4e47-ad51-51e973962aa1 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:33.745652 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03297b-096d-4c2f-95b2-6f22bd84c0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:33.746810 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03297b-096d-4c2f-95b2-6f22bd84c0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:33.747124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03297b-096d-4c2f-95b2-6f22bd84c0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:33.748008 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03297b-096d-4c2f-95b2-6f22bd84c0e2 admin admin] PATH_INFO: `/v3/projects/72acefad68e9438e8c025c229ac36607` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:33.749517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03297b-096d-4c2f-95b2-6f22bd84c0e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72acefad68e9438e8c025c229ac36607)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:33.751526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03297b-096d-4c2f-95b2-6f22bd84c0e2 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:33.789853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:33.803664 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9a03297b-096d-4c2f-95b2-6f22bd84c0e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72acefad68e9438e8c025c229ac36607', 'request_id': 'req-9a03297b-096d-4c2f-95b2-6f22bd84c0e2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:33.810366 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9a03297b-096d-4c2f-95b2-6f22bd84c0e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72acefad68e9438e8c025c229ac36607', 'request_id': 'req-9a03297b-096d-4c2f-95b2-6f22bd84c0e2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:33.816639 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9a03297b-096d-4c2f-95b2-6f22bd84c0e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72acefad68e9438e8c025c229ac36607', 'request_id': 'req-9a03297b-096d-4c2f-95b2-6f22bd84c0e2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:33.831699 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9a03297b-096d-4c2f-95b2-6f22bd84c0e2 admin admin] The token cache is being invalidate because project 72acefad68e9438e8c025c229ac36607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:46:33.832007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9a03297b-096d-4c2f-95b2-6f22bd84c0e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:33.832629 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9a03297b-096d-4c2f-95b2-6f22bd84c0e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:46:33.834335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1232/2470] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:46:33 2022] DELETE /identity/v3/projects/72acefad68e9438e8c025c229ac36607 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:33.839458 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff2c6a-2b91-4727-944c-8be20e7bf14b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:33.839748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff2c6a-2b91-4727-944c-8be20e7bf14b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:33.840042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff2c6a-2b91-4727-944c-8be20e7bf14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:33.858213 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-4aff2c6a-2b91-4727-944c-8be20e7bf14b None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:33.859448 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4aff2c6a-2b91-4727-944c-8be20e7bf14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:33.919871 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1239/2471] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:33.931687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a078d4-37ab-460e-b2b7-cfa5a5021e8f None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:33.931980 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a078d4-37ab-460e-b2b7-cfa5a5021e8f None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:33.932244 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a078d4-37ab-460e-b2b7-cfa5a5021e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:33.952662 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-33a078d4-37ab-460e-b2b7-cfa5a5021e8f None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:33.953911 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-33a078d4-37ab-460e-b2b7-cfa5a5021e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:33.963000 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1233/2472] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:33.967622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff2c6a-2b91-4727-944c-8be20e7bf14b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:33.977960 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff2c6a-2b91-4727-944c-8be20e7bf14b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:33.984108 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f9a8b-9baa-4002-841b-1203f4faa580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:33.985037 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f9a8b-9baa-4002-841b-1203f4faa580 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:33.985367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f9a8b-9baa-4002-841b-1203f4faa580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:33.985724 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f9a8b-9baa-4002-841b-1203f4faa580 admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:33.986480 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f9a8b-9baa-4002-841b-1203f4faa580 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:46:33.986779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f9a8b-9baa-4002-841b-1203f4faa580 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:33.988468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f9a8b-9baa-4002-841b-1203f4faa580 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:34.002596 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1240/2473] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:46:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:46:34.007204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a078d4-37ab-460e-b2b7-cfa5a5021e8f None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:34.022257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a078d4-37ab-460e-b2b7-cfa5a5021e8f None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:34.027570 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8350e6-9c24-4c99-8862-0df114f83aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:34.028241 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8350e6-9c24-4c99-8862-0df114f83aaa admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:34.028518 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8350e6-9c24-4c99-8862-0df114f83aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:34.028785 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8350e6-9c24-4c99-8862-0df114f83aaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:34.029494 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8350e6-9c24-4c99-8862-0df114f83aaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:34.031625 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8350e6-9c24-4c99-8862-0df114f83aaa admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:34.067882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1234/2474] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:46:34 2022] POST /identity/v3/projects => generated 389 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:46:34.072274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f9a8b-9baa-4002-841b-1203f4faa580 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:34.086243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f9a8b-9baa-4002-841b-1203f4faa580 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:34.147280 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d25243-53f0-4473-9a0d-c43ec8f31dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:34.147969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d25243-53f0-4473-9a0d-c43ec8f31dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:34.148253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d25243-53f0-4473-9a0d-c43ec8f31dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:34.148528 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d25243-53f0-4473-9a0d-c43ec8f31dd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:34.149228 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d25243-53f0-4473-9a0d-c43ec8f31dd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:34.151193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d25243-53f0-4473-9a0d-c43ec8f31dd7 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:34.173459 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1241/2475] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:46:34 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:46:34.177763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8350e6-9c24-4c99-8862-0df114f83aaa admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:34.189021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8350e6-9c24-4c99-8862-0df114f83aaa admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:34.194925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a189cb-8180-44c3-a59a-5be571d789cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:34.195703 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a189cb-8180-44c3-a59a-5be571d789cc admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:34.195988 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a189cb-8180-44c3-a59a-5be571d789cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:34.196263 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a189cb-8180-44c3-a59a-5be571d789cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:34.196997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a189cb-8180-44c3-a59a-5be571d789cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:34.198621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a189cb-8180-44c3-a59a-5be571d789cc admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:34.211059 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1235/2476] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:46:34.214302 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d25243-53f0-4473-9a0d-c43ec8f31dd7 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:34.224782 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d25243-53f0-4473-9a0d-c43ec8f31dd7 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:34.230690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9243a5b-fca9-4446-b90b-0c1071ef2246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:34.231365 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9243a5b-fca9-4446-b90b-0c1071ef2246 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:34.231668 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9243a5b-fca9-4446-b90b-0c1071ef2246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:34.231978 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9243a5b-fca9-4446-b90b-0c1071ef2246 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:34.232828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9243a5b-fca9-4446-b90b-0c1071ef2246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:34.234472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9243a5b-fca9-4446-b90b-0c1071ef2246 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:34.245509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1242/2477] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:46:34.249838 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a189cb-8180-44c3-a59a-5be571d789cc admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:34.260299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a189cb-8180-44c3-a59a-5be571d789cc admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:34.266004 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93abf4-c482-4eaf-b13b-c2f678ab31bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:34.266896 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93abf4-c482-4eaf-b13b-c2f678ab31bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:34.267216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93abf4-c482-4eaf-b13b-c2f678ab31bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:34.267494 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93abf4-c482-4eaf-b13b-c2f678ab31bc admin admin] PATH_INFO: `/v3/projects/90c0af11e48a488d9806b6125e0afa60/users/aae7b76ac2e14933af40f75225cc840f/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:34.282989 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93abf4-c482-4eaf-b13b-c2f678ab31bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c0af11e48a488d9806b6125e0afa60, user_id=aae7b76ac2e14933af40f75225cc840f, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:34.286099 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93abf4-c482-4eaf-b13b-c2f678ab31bc admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:34.301647 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1236/2478] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:46:34 2022] PUT /identity/v3/projects/90c0af11e48a488d9806b6125e0afa60/users/aae7b76ac2e14933af40f75225cc840f/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:46:34.313876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9f022-5de4-45ba-984a-de9126b57a3d None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:34.313876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9f022-5de4-45ba-984a-de9126b57a3d None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:34.313876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9f022-5de4-45ba-984a-de9126b57a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:34.344932 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-adb9f022-5de4-45ba-984a-de9126b57a3d None None] MFA Rules not processed for user `aae7b76ac2e14933af40f75225cc840f`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:46:34.345834 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-adb9f022-5de4-45ba-984a-de9126b57a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:34.397480 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:34.434667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1243/2479] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:46:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:46:34.442726 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93abf4-c482-4eaf-b13b-c2f678ab31bc admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:34.444677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ed93abf4-c482-4eaf-b13b-c2f678ab31bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:34.506484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93abf4-c482-4eaf-b13b-c2f678ab31bc admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:34.510719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664520a-9e2f-4b53-8f62-f5b1debc44f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:34.511432 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664520a-9e2f-4b53-8f62-f5b1debc44f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:34.511741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664520a-9e2f-4b53-8f62-f5b1debc44f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:34.512014 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664520a-9e2f-4b53-8f62-f5b1debc44f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:34.520516 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664520a-9e2f-4b53-8f62-f5b1debc44f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:34.522062 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664520a-9e2f-4b53-8f62-f5b1debc44f9 service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:34.531305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1237/2480] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:46:44.427102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=7 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:44.527493 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:46:53.498518 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9f022-5de4-45ba-984a-de9126b57a3d None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:46:53.500099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-adb9f022-5de4-45ba-984a-de9126b57a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:53.574688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb9f022-5de4-45ba-984a-de9126b57a3d None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:46:53.580581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37458b6-bb82-455a-991b-0348e760e42b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:46:53.581243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37458b6-bb82-455a-991b-0348e760e42b service placement] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:46:53.581521 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37458b6-bb82-455a-991b-0348e760e42b service placement] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:46:53.581792 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37458b6-bb82-455a-991b-0348e760e42b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:46:53.583733 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c37458b6-bb82-455a-991b-0348e760e42b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:46:53.595908 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37458b6-bb82-455a-991b-0348e760e42b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:46:53.597568 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37458b6-bb82-455a-991b-0348e760e42b service placement] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:46:53.607455 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1244/2481] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:46:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:47:03.603097 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=32 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:47:04.722478 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664520a-9e2f-4b53-8f62-f5b1debc44f9 service cinder] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:04.724260 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0664520a-9e2f-4b53-8f62-f5b1debc44f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:04.799120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664520a-9e2f-4b53-8f62-f5b1debc44f9 service cinder] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:04.804269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f996f63-e094-428d-8bc2-754037bc6a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:04.805227 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f996f63-e094-428d-8bc2-754037bc6a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:04.805503 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f996f63-e094-428d-8bc2-754037bc6a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:04.805787 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f996f63-e094-428d-8bc2-754037bc6a8e admin admin] PATH_INFO: `/v3/users/c7f987f5904d45a5bd6d3f991b9a406f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:04.807061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f996f63-e094-428d-8bc2-754037bc6a8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7f987f5904d45a5bd6d3f991b9a406f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:04.809039 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f996f63-e094-428d-8bc2-754037bc6a8e admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:04.863398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-8f996f63-e094-428d-8bc2-754037bc6a8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7f987f5904d45a5bd6d3f991b9a406f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:04.875904 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-8f996f63-e094-428d-8bc2-754037bc6a8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7f987f5904d45a5bd6d3f991b9a406f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:04.882503 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-8f996f63-e094-428d-8bc2-754037bc6a8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7f987f5904d45a5bd6d3f991b9a406f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:04.883224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-8f996f63-e094-428d-8bc2-754037bc6a8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7f987f5904d45a5bd6d3f991b9a406f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:04.883835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-8f996f63-e094-428d-8bc2-754037bc6a8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7f987f5904d45a5bd6d3f991b9a406f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:04.898608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1238/2482] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:47:04 2022] DELETE /identity/v3/users/c7f987f5904d45a5bd6d3f991b9a406f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:47:04.903292 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603744e-a095-48f2-b07c-0faff2a6eec4 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:04.903861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603744e-a095-48f2-b07c-0faff2a6eec4 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:04.903972 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603744e-a095-48f2-b07c-0faff2a6eec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:04.921730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-2603744e-a095-48f2-b07c-0faff2a6eec4 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:47:04.922914 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2603744e-a095-48f2-b07c-0faff2a6eec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:04.986429 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1245/2483] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:47:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:47:04.996387 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f996f63-e094-428d-8bc2-754037bc6a8e admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:04.997986 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8f996f63-e094-428d-8bc2-754037bc6a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:05.060665 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f996f63-e094-428d-8bc2-754037bc6a8e admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:05.070522 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa22337-da67-49a6-88fb-8e45463dfc75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:05.071643 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa22337-da67-49a6-88fb-8e45463dfc75 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:05.071790 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa22337-da67-49a6-88fb-8e45463dfc75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:05.072031 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa22337-da67-49a6-88fb-8e45463dfc75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:05.084729 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa22337-da67-49a6-88fb-8e45463dfc75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:05.086194 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa22337-da67-49a6-88fb-8e45463dfc75 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:05.095970 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1239/2484] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:47:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:47:05.587142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603744e-a095-48f2-b07c-0faff2a6eec4 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:05.588628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2603744e-a095-48f2-b07c-0faff2a6eec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:05.600827 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603744e-a095-48f2-b07c-0faff2a6eec4 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:05.605592 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5a1a1-b987-49bd-a125-1b20752f6d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:05.606525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5a1a1-b987-49bd-a125-1b20752f6d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:05.606811 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5a1a1-b987-49bd-a125-1b20752f6d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:05.607290 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5a1a1-b987-49bd-a125-1b20752f6d0b admin admin] PATH_INFO: `/v3/projects/b66c6ff9fade4d528d45545fc4806792` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:05.608511 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5a1a1-b987-49bd-a125-1b20752f6d0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b66c6ff9fade4d528d45545fc4806792)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:05.610452 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5a1a1-b987-49bd-a125-1b20752f6d0b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:05.667896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f2c5a1a1-b987-49bd-a125-1b20752f6d0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b66c6ff9fade4d528d45545fc4806792', 'request_id': 'req-f2c5a1a1-b987-49bd-a125-1b20752f6d0b'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:05.673830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f2c5a1a1-b987-49bd-a125-1b20752f6d0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b66c6ff9fade4d528d45545fc4806792', 'request_id': 'req-f2c5a1a1-b987-49bd-a125-1b20752f6d0b'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:05.682664 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f2c5a1a1-b987-49bd-a125-1b20752f6d0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b66c6ff9fade4d528d45545fc4806792', 'request_id': 'req-f2c5a1a1-b987-49bd-a125-1b20752f6d0b'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:05.697908 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f2c5a1a1-b987-49bd-a125-1b20752f6d0b admin admin] The token cache is being invalidate because project b66c6ff9fade4d528d45545fc4806792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:47:05.698260 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f2c5a1a1-b987-49bd-a125-1b20752f6d0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:05.698918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f2c5a1a1-b987-49bd-a125-1b20752f6d0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:05.700726 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1246/2485] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:47:05 2022] DELETE /identity/v3/projects/b66c6ff9fade4d528d45545fc4806792 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:47:08.522359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa22337-da67-49a6-88fb-8e45463dfc75 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:08.524077 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-2aa22337-da67-49a6-88fb-8e45463dfc75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:08.539471 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa22337-da67-49a6-88fb-8e45463dfc75 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:08.559815 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:47:08.601102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f4b46-17a5-4689-a9d4-7ce960d9ff1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:08.601981 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f4b46-17a5-4689-a9d4-7ce960d9ff1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:08.602269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f4b46-17a5-4689-a9d4-7ce960d9ff1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:08.602538 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f4b46-17a5-4689-a9d4-7ce960d9ff1d admin admin] PATH_INFO: `/v3/users/aae7b76ac2e14933af40f75225cc840f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:08.603766 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f4b46-17a5-4689-a9d4-7ce960d9ff1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aae7b76ac2e14933af40f75225cc840f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:08.605744 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f4b46-17a5-4689-a9d4-7ce960d9ff1d admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:08.655042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-601f4b46-17a5-4689-a9d4-7ce960d9ff1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aae7b76ac2e14933af40f75225cc840f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:08.667314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-601f4b46-17a5-4689-a9d4-7ce960d9ff1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aae7b76ac2e14933af40f75225cc840f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:08.674373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-601f4b46-17a5-4689-a9d4-7ce960d9ff1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aae7b76ac2e14933af40f75225cc840f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:08.675097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-601f4b46-17a5-4689-a9d4-7ce960d9ff1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aae7b76ac2e14933af40f75225cc840f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:08.675721 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-601f4b46-17a5-4689-a9d4-7ce960d9ff1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aae7b76ac2e14933af40f75225cc840f'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:08.690715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1240/2486] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:47:08 2022] DELETE /identity/v3/users/aae7b76ac2e14933af40f75225cc840f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:47:08.695497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544c376-e849-470f-a7f1-07fda16278ca None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:08.695962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544c376-e849-470f-a7f1-07fda16278ca None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:08.696065 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544c376-e849-470f-a7f1-07fda16278ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:08.715810 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-b544c376-e849-470f-a7f1-07fda16278ca None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:47:08.717024 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b544c376-e849-470f-a7f1-07fda16278ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:08.779628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1247/2487] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:47:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:47:08.789747 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f4b46-17a5-4689-a9d4-7ce960d9ff1d admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:08.791286 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-601f4b46-17a5-4689-a9d4-7ce960d9ff1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:08.852426 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f4b46-17a5-4689-a9d4-7ce960d9ff1d admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:08.856106 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ccead-26e8-4524-afc2-539a26342ab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:08.856760 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ccead-26e8-4524-afc2-539a26342ab6 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:08.857038 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ccead-26e8-4524-afc2-539a26342ab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:08.857312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ccead-26e8-4524-afc2-539a26342ab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:08.867664 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ccead-26e8-4524-afc2-539a26342ab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:08.869185 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ccead-26e8-4524-afc2-539a26342ab6 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:08.881591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1241/2488] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:47:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:47:09.149659 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544c376-e849-470f-a7f1-07fda16278ca None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:09.151318 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b544c376-e849-470f-a7f1-07fda16278ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:09.163962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544c376-e849-470f-a7f1-07fda16278ca None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:09.169382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b5be1-5f49-4684-a5f4-d2ef0ddedcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:09.170276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b5be1-5f49-4684-a5f4-d2ef0ddedcb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:09.170688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b5be1-5f49-4684-a5f4-d2ef0ddedcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:09.170971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b5be1-5f49-4684-a5f4-d2ef0ddedcb4 admin admin] PATH_INFO: `/v3/projects/90c0af11e48a488d9806b6125e0afa60` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:09.172397 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b5be1-5f49-4684-a5f4-d2ef0ddedcb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90c0af11e48a488d9806b6125e0afa60)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:09.174509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b5be1-5f49-4684-a5f4-d2ef0ddedcb4 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:09.198578 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:47:09.227372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-2a2b5be1-5f49-4684-a5f4-d2ef0ddedcb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90c0af11e48a488d9806b6125e0afa60', 'request_id': 'req-2a2b5be1-5f49-4684-a5f4-d2ef0ddedcb4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:09.233222 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-2a2b5be1-5f49-4684-a5f4-d2ef0ddedcb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90c0af11e48a488d9806b6125e0afa60', 'request_id': 'req-2a2b5be1-5f49-4684-a5f4-d2ef0ddedcb4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:09.241184 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-2a2b5be1-5f49-4684-a5f4-d2ef0ddedcb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90c0af11e48a488d9806b6125e0afa60', 'request_id': 'req-2a2b5be1-5f49-4684-a5f4-d2ef0ddedcb4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:09.252296 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-2a2b5be1-5f49-4684-a5f4-d2ef0ddedcb4 admin admin] The token cache is being invalidate because project 90c0af11e48a488d9806b6125e0afa60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:47:09.252583 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-2a2b5be1-5f49-4684-a5f4-d2ef0ddedcb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:09.253256 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-2a2b5be1-5f49-4684-a5f4-d2ef0ddedcb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:09.254836 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1248/2489] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:47:09 2022] DELETE /identity/v3/projects/90c0af11e48a488d9806b6125e0afa60 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:47:12.413617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ccead-26e8-4524-afc2-539a26342ab6 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:12.415211 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-cf3ccead-26e8-4524-afc2-539a26342ab6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:12.428988 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ccead-26e8-4524-afc2-539a26342ab6 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:12.479927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:47:12.497978 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61f683-8245-431a-bbf8-e09fc0feffc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:12.498852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61f683-8245-431a-bbf8-e09fc0feffc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:12.499161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61f683-8245-431a-bbf8-e09fc0feffc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:12.499439 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61f683-8245-431a-bbf8-e09fc0feffc9 admin admin] PATH_INFO: `/v3/users/2efb609ec03343c8be1f4cdb977c48c8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:12.500643 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61f683-8245-431a-bbf8-e09fc0feffc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2efb609ec03343c8be1f4cdb977c48c8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:12.502598 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61f683-8245-431a-bbf8-e09fc0feffc9 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:12.553438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-6c61f683-8245-431a-bbf8-e09fc0feffc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2efb609ec03343c8be1f4cdb977c48c8'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:12.566127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-6c61f683-8245-431a-bbf8-e09fc0feffc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2efb609ec03343c8be1f4cdb977c48c8'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:12.573677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-6c61f683-8245-431a-bbf8-e09fc0feffc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2efb609ec03343c8be1f4cdb977c48c8'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:12.574418 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-6c61f683-8245-431a-bbf8-e09fc0feffc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2efb609ec03343c8be1f4cdb977c48c8'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:12.575155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-6c61f683-8245-431a-bbf8-e09fc0feffc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2efb609ec03343c8be1f4cdb977c48c8'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:12.589839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1242/2490] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:47:12 2022] DELETE /identity/v3/users/2efb609ec03343c8be1f4cdb977c48c8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:47:12.594445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de83507-f1f7-482f-8bc9-da8f9c3f47a9 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:12.594919 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de83507-f1f7-482f-8bc9-da8f9c3f47a9 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:12.595066 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de83507-f1f7-482f-8bc9-da8f9c3f47a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:12.615850 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-5de83507-f1f7-482f-8bc9-da8f9c3f47a9 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:47:12.617155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5de83507-f1f7-482f-8bc9-da8f9c3f47a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:12.681476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1249/2491] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:47:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:47:12.691516 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61f683-8245-431a-bbf8-e09fc0feffc9 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:12.693109 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-6c61f683-8245-431a-bbf8-e09fc0feffc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:12.752116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61f683-8245-431a-bbf8-e09fc0feffc9 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:12.756425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c15fe-c1c6-4329-a8ff-865e6014605f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:12.757066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c15fe-c1c6-4329-a8ff-865e6014605f service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:12.757343 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c15fe-c1c6-4329-a8ff-865e6014605f service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:12.757614 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c15fe-c1c6-4329-a8ff-865e6014605f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:12.767409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c15fe-c1c6-4329-a8ff-865e6014605f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:12.768835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c15fe-c1c6-4329-a8ff-865e6014605f service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:12.778525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1243/2492] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:47:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:47:13.069097 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de83507-f1f7-482f-8bc9-da8f9c3f47a9 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:13.071332 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5de83507-f1f7-482f-8bc9-da8f9c3f47a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:13.085388 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de83507-f1f7-482f-8bc9-da8f9c3f47a9 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:13.090424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b4896-fe5a-4490-a001-62b47e08c207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:13.092159 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b4896-fe5a-4490-a001-62b47e08c207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:13.092447 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b4896-fe5a-4490-a001-62b47e08c207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:13.092724 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b4896-fe5a-4490-a001-62b47e08c207 admin admin] PATH_INFO: `/v3/projects/0f555c1188454dd2a5132e9c80a1e262` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:13.094103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b4896-fe5a-4490-a001-62b47e08c207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f555c1188454dd2a5132e9c80a1e262)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:13.096161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b4896-fe5a-4490-a001-62b47e08c207 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:13.145473 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3a4b4896-fe5a-4490-a001-62b47e08c207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f555c1188454dd2a5132e9c80a1e262', 'request_id': 'req-3a4b4896-fe5a-4490-a001-62b47e08c207'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:13.150849 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3a4b4896-fe5a-4490-a001-62b47e08c207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f555c1188454dd2a5132e9c80a1e262', 'request_id': 'req-3a4b4896-fe5a-4490-a001-62b47e08c207'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:13.159623 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3a4b4896-fe5a-4490-a001-62b47e08c207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f555c1188454dd2a5132e9c80a1e262', 'request_id': 'req-3a4b4896-fe5a-4490-a001-62b47e08c207'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:13.170722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3a4b4896-fe5a-4490-a001-62b47e08c207 admin admin] The token cache is being invalidate because project 0f555c1188454dd2a5132e9c80a1e262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:47:13.171034 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3a4b4896-fe5a-4490-a001-62b47e08c207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:13.171659 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3a4b4896-fe5a-4490-a001-62b47e08c207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:47:13.173342 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1250/2493] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:47:13 2022] DELETE /identity/v3/projects/0f555c1188454dd2a5132e9c80a1e262 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:47:13.178609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95015d-ae74-434c-bd45-5de7999da828 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:13.178842 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95015d-ae74-434c-bd45-5de7999da828 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:13.179156 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95015d-ae74-434c-bd45-5de7999da828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:13.196708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-2e95015d-ae74-434c-bd45-5de7999da828 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:47:13.197902 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-2e95015d-ae74-434c-bd45-5de7999da828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:13.261610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1244/2494] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:47:13.270138 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a03a7-7b45-4045-8756-b8a4706cd29d None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:13.270428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a03a7-7b45-4045-8756-b8a4706cd29d None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:13.270706 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a03a7-7b45-4045-8756-b8a4706cd29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:13.289944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-251a03a7-7b45-4045-8756-b8a4706cd29d None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:47:13.291214 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-251a03a7-7b45-4045-8756-b8a4706cd29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:13.300893 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1251/2495] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:47:13.305125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95015d-ae74-434c-bd45-5de7999da828 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:13.315361 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95015d-ae74-434c-bd45-5de7999da828 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:13.321537 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d498b-7ad0-4596-9b45-65f2f5f92bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:13.322195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d498b-7ad0-4596-9b45-65f2f5f92bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:13.322474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d498b-7ad0-4596-9b45-65f2f5f92bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:13.322743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d498b-7ad0-4596-9b45-65f2f5f92bd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:13.323505 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d498b-7ad0-4596-9b45-65f2f5f92bd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:47:13.323810 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d498b-7ad0-4596-9b45-65f2f5f92bd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:13.325418 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d498b-7ad0-4596-9b45-65f2f5f92bd8 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:13.326879 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:47:13.338656 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1245/2496] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:47:13.343531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a03a7-7b45-4045-8756-b8a4706cd29d None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:13.353082 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a03a7-7b45-4045-8756-b8a4706cd29d None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:13.358233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd650228-84be-4dce-a024-dba695de564b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:13.358886 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd650228-84be-4dce-a024-dba695de564b admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:13.359204 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd650228-84be-4dce-a024-dba695de564b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:13.359482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd650228-84be-4dce-a024-dba695de564b admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:13.360261 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd650228-84be-4dce-a024-dba695de564b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:13.362203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd650228-84be-4dce-a024-dba695de564b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:13.389182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1252/2497] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:47:13 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:47:13.393614 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d498b-7ad0-4596-9b45-65f2f5f92bd8 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:13.404695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d498b-7ad0-4596-9b45-65f2f5f92bd8 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:13.476657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a731a2-12d3-42e4-a8bd-2e0ed936919e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:13.477312 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a731a2-12d3-42e4-a8bd-2e0ed936919e admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:13.477561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a731a2-12d3-42e4-a8bd-2e0ed936919e admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:13.477845 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a731a2-12d3-42e4-a8bd-2e0ed936919e admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:13.478853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a731a2-12d3-42e4-a8bd-2e0ed936919e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:13.481909 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a731a2-12d3-42e4-a8bd-2e0ed936919e admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:13.504321 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1246/2498] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:47:13 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:47:13.508502 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd650228-84be-4dce-a024-dba695de564b admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:13.513990 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:47:13.520730 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd650228-84be-4dce-a024-dba695de564b admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:13.526607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b2978-038e-46c8-b5fe-10b6485d5db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:13.527309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b2978-038e-46c8-b5fe-10b6485d5db3 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:13.527615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b2978-038e-46c8-b5fe-10b6485d5db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:13.527892 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b2978-038e-46c8-b5fe-10b6485d5db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:13.528630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b2978-038e-46c8-b5fe-10b6485d5db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:13.530245 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b2978-038e-46c8-b5fe-10b6485d5db3 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:13.541669 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1253/2499] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:47:13.545607 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a731a2-12d3-42e4-a8bd-2e0ed936919e admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:13.555793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a731a2-12d3-42e4-a8bd-2e0ed936919e admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:13.560997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd1b14-e082-467a-a405-f716929b11b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:13.561656 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd1b14-e082-467a-a405-f716929b11b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:13.561934 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd1b14-e082-467a-a405-f716929b11b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:13.562239 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd1b14-e082-467a-a405-f716929b11b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:13.562953 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd1b14-e082-467a-a405-f716929b11b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:13.564627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd1b14-e082-467a-a405-f716929b11b1 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:13.575122 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1247/2500] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:47:13.578862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b2978-038e-46c8-b5fe-10b6485d5db3 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:13.590944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b2978-038e-46c8-b5fe-10b6485d5db3 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:13.595866 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384afb5-f59d-46f7-af88-e3cf8ac2f81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:13.596757 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384afb5-f59d-46f7-af88-e3cf8ac2f81b admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:13.597033 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384afb5-f59d-46f7-af88-e3cf8ac2f81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:13.597301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384afb5-f59d-46f7-af88-e3cf8ac2f81b admin admin] PATH_INFO: `/v3/projects/c1c0c1c0f91147b98e49acb05e14d2d2/users/22b285ea44c0494d9fa611df20d66a27/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:13.612128 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384afb5-f59d-46f7-af88-e3cf8ac2f81b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c0c1c0f91147b98e49acb05e14d2d2, user_id=22b285ea44c0494d9fa611df20d66a27, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:13.614803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384afb5-f59d-46f7-af88-e3cf8ac2f81b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:13.629335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1254/2501] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:47:13 2022] PUT /identity/v3/projects/c1c0c1c0f91147b98e49acb05e14d2d2/users/22b285ea44c0494d9fa611df20d66a27/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:47:13.634468 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5f417-f267-487a-934e-3a460897eb30 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:13.634788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5f417-f267-487a-934e-3a460897eb30 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:13.635052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5f417-f267-487a-934e-3a460897eb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:13.652960 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-2cc5f417-f267-487a-934e-3a460897eb30 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:47:13.654155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-2cc5f417-f267-487a-934e-3a460897eb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:13.713531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1248/2502] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:47:13.723670 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384afb5-f59d-46f7-af88-e3cf8ac2f81b admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:13.725501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c384afb5-f59d-46f7-af88-e3cf8ac2f81b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:13.783176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384afb5-f59d-46f7-af88-e3cf8ac2f81b admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:13.787499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a3406-8090-4b3c-8d42-cc67057a9a32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:13.788143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a3406-8090-4b3c-8d42-cc67057a9a32 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:13.788427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a3406-8090-4b3c-8d42-cc67057a9a32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:13.788698 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a3406-8090-4b3c-8d42-cc67057a9a32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:13.798060 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a3406-8090-4b3c-8d42-cc67057a9a32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:13.799551 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a3406-8090-4b3c-8d42-cc67057a9a32 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:13.808971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1255/2503] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:47:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:47:17.522357 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45b2c0-7531-413f-a5b2-ee8e9906fddd None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:17.522923 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45b2c0-7531-413f-a5b2-ee8e9906fddd None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:17.522923 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45b2c0-7531-413f-a5b2-ee8e9906fddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:17.553014 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-ae45b2c0-7531-413f-a5b2-ee8e9906fddd None None] MFA Rules not processed for user `22b285ea44c0494d9fa611df20d66a27`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:47:17.554175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ae45b2c0-7531-413f-a5b2-ee8e9906fddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:17.625400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1249/2504] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:47:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:47:17.630316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a3406-8090-4b3c-8d42-cc67057a9a32 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:17.640522 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a3406-8090-4b3c-8d42-cc67057a9a32 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:17.645384 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fcd63-20ed-499b-9e26-9c0ff049d037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:17.646140 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fcd63-20ed-499b-9e26-9c0ff049d037 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:17.646484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fcd63-20ed-499b-9e26-9c0ff049d037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:17.646844 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fcd63-20ed-499b-9e26-9c0ff049d037 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:17.647623 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900fcd63-20ed-499b-9e26-9c0ff049d037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:17.649597 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900fcd63-20ed-499b-9e26-9c0ff049d037 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:17.679753 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1256/2505] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:47:17 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:47:17.683709 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45b2c0-7531-413f-a5b2-ee8e9906fddd None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:17.689727 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:47:17.697112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45b2c0-7531-413f-a5b2-ee8e9906fddd None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:17.761361 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f7bf6-ec07-4cb3-96fa-15becf3b3bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:17.762151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f7bf6-ec07-4cb3-96fa-15becf3b3bed admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:17.762502 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f7bf6-ec07-4cb3-96fa-15becf3b3bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:17.762789 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f7bf6-ec07-4cb3-96fa-15becf3b3bed admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:17.763598 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f7bf6-ec07-4cb3-96fa-15becf3b3bed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:17.765609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f7bf6-ec07-4cb3-96fa-15becf3b3bed admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:17.794441 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1250/2506] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:47:17 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:47:17.798242 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fcd63-20ed-499b-9e26-9c0ff049d037 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:17.834356 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fcd63-20ed-499b-9e26-9c0ff049d037 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:17.839901 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c4e73-e068-4caf-b5a2-0aba4efe5f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:17.840518 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c4e73-e068-4caf-b5a2-0aba4efe5f90 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:17.840802 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c4e73-e068-4caf-b5a2-0aba4efe5f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:17.841088 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c4e73-e068-4caf-b5a2-0aba4efe5f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:17.841815 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c4e73-e068-4caf-b5a2-0aba4efe5f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:17.843589 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c4e73-e068-4caf-b5a2-0aba4efe5f90 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:17.854956 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1257/2507] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:47:17.860288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f7bf6-ec07-4cb3-96fa-15becf3b3bed admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:17.874926 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f7bf6-ec07-4cb3-96fa-15becf3b3bed admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:17.882919 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a8fbe-0e55-48f5-a15e-6efe3eae7a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:17.884003 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a8fbe-0e55-48f5-a15e-6efe3eae7a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:17.884203 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a8fbe-0e55-48f5-a15e-6efe3eae7a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:17.884431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a8fbe-0e55-48f5-a15e-6efe3eae7a41 admin admin] PATH_INFO: `/v3/projects/108552a2639a4fe2ab275355ca6f9e00/users/053c267e63c041d3808e53950cc67ab7/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:17.901635 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a8fbe-0e55-48f5-a15e-6efe3eae7a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=108552a2639a4fe2ab275355ca6f9e00, user_id=053c267e63c041d3808e53950cc67ab7, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:17.905452 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a8fbe-0e55-48f5-a15e-6efe3eae7a41 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:17.922674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1251/2508] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:47:17 2022] PUT /identity/v3/projects/108552a2639a4fe2ab275355ca6f9e00/users/053c267e63c041d3808e53950cc67ab7/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:47:17.926701 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c4e73-e068-4caf-b5a2-0aba4efe5f90 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:17.937475 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c4e73-e068-4caf-b5a2-0aba4efe5f90 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:18.013745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf584a7e-acf6-42a9-b4a1-aff40f2c4036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:18.014439 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf584a7e-acf6-42a9-b4a1-aff40f2c4036 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:18.014866 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf584a7e-acf6-42a9-b4a1-aff40f2c4036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:18.014956 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf584a7e-acf6-42a9-b4a1-aff40f2c4036 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:18.015734 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf584a7e-acf6-42a9-b4a1-aff40f2c4036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:18.017405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf584a7e-acf6-42a9-b4a1-aff40f2c4036 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:18.028189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1258/2509] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:47:18.032531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a8fbe-0e55-48f5-a15e-6efe3eae7a41 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:18.044207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a8fbe-0e55-48f5-a15e-6efe3eae7a41 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:18.050133 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b93f5-86df-4572-b6c7-6dd056763f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:18.051067 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b93f5-86df-4572-b6c7-6dd056763f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:18.051530 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b93f5-86df-4572-b6c7-6dd056763f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:18.051807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b93f5-86df-4572-b6c7-6dd056763f6d admin admin] PATH_INFO: `/v3/projects/108552a2639a4fe2ab275355ca6f9e00/users/053c267e63c041d3808e53950cc67ab7/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:18.053925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b93f5-86df-4572-b6c7-6dd056763f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=108552a2639a4fe2ab275355ca6f9e00, user_id=053c267e63c041d3808e53950cc67ab7, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:18.056853 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b93f5-86df-4572-b6c7-6dd056763f6d admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:18.069036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1252/2510] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:47:18 2022] PUT /identity/v3/projects/108552a2639a4fe2ab275355ca6f9e00/users/053c267e63c041d3808e53950cc67ab7/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:47:22.407355 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa0c18-4ba6-4577-b1ad-3d8825d5e5f5 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:22.407879 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa0c18-4ba6-4577-b1ad-3d8825d5e5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:22.407977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa0c18-4ba6-4577-b1ad-3d8825d5e5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:22.434823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-99aa0c18-4ba6-4577-b1ad-3d8825d5e5f5 None None] MFA Rules not processed for user `053c267e63c041d3808e53950cc67ab7`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:47:22.435996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-99aa0c18-4ba6-4577-b1ad-3d8825d5e5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:22.457229 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:47:22.507407 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1259/2511] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:47:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:47:22.520084 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b93f5-86df-4572-b6c7-6dd056763f6d admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:22.521995 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-fd6b93f5-86df-4572-b6c7-6dd056763f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:22.586954 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b93f5-86df-4572-b6c7-6dd056763f6d admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:22.590472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab61b5-a021-4bef-8dea-5bc24aeb1930 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2707c6967a0a419a98ae142ff1d1a961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:22.591161 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab61b5-a021-4bef-8dea-5bc24aeb1930 service glance] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:22.591442 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab61b5-a021-4bef-8dea-5bc24aeb1930 service glance] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:22.591713 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab61b5-a021-4bef-8dea-5bc24aeb1930 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:22.675204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab61b5-a021-4bef-8dea-5bc24aeb1930 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:22.677653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab61b5-a021-4bef-8dea-5bc24aeb1930 service glance] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:22.689007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1253/2512] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:47:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:47:22.847740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e756d5-a618-45a6-882c-d64b9c220316 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:22.848036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e756d5-a618-45a6-882c-d64b9c220316 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:22.848310 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e756d5-a618-45a6-882c-d64b9c220316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:22.868271 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-f1e756d5-a618-45a6-882c-d64b9c220316 None None] MFA Rules not processed for user `4d04998b636649d98103b1cd5f3a0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:47:22.869461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f1e756d5-a618-45a6-882c-d64b9c220316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:22.945755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1260/2513] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jul 21 05:47:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:47:22.996718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab61b5-a021-4bef-8dea-5bc24aeb1930 service glance] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:22.999147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f9ab61b5-a021-4bef-8dea-5bc24aeb1930 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:23.013554 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:47:23.110250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab61b5-a021-4bef-8dea-5bc24aeb1930 service glance] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:23.128137 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d6a46-34a1-42c3-aa04-c4a393debbe9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61b61188b5b94a51ba1ae1a88b78b89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:23.128844 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d6a46-34a1-42c3-aa04-c4a393debbe9 service swift] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:23.129141 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d6a46-34a1-42c3-aa04-c4a393debbe9 service swift] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:23.129420 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d6a46-34a1-42c3-aa04-c4a393debbe9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:23.144766 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d6a46-34a1-42c3-aa04-c4a393debbe9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:23.147712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d6a46-34a1-42c3-aa04-c4a393debbe9 service swift] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:23.156617 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1254/2514] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jul 21 05:47:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 05:47:32.016878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576c4ed-51d8-40a3-9bbe-8271e3780361 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:32.017456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576c4ed-51d8-40a3-9bbe-8271e3780361 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:32.017456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576c4ed-51d8-40a3-9bbe-8271e3780361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:32.038660 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-3576c4ed-51d8-40a3-9bbe-8271e3780361 None None] MFA Rules not processed for user `22b285ea44c0494d9fa611df20d66a27`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:47:32.039930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-3576c4ed-51d8-40a3-9bbe-8271e3780361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:32.047488 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1261/2515] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:47:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:47:33.153451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=28 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:47:36.519347 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f0ee5-1977-4e99-b4a4-0ac62ead1c2f None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:36.519940 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f0ee5-1977-4e99-b4a4-0ac62ead1c2f None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:36.520036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f0ee5-1977-4e99-b4a4-0ac62ead1c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:36.539304 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-b35f0ee5-1977-4e99-b4a4-0ac62ead1c2f None None] MFA Rules not processed for user `4d04998b636649d98103b1cd5f3a0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:47:36.540825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b35f0ee5-1977-4e99-b4a4-0ac62ead1c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:47:36.549447 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1255/2516] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jul 21 05:47:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:47:36.561695 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3576c4ed-51d8-40a3-9bbe-8271e3780361 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:47:36.573914 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3576c4ed-51d8-40a3-9bbe-8271e3780361 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:47:36.578722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d67cf5-0ae1-4e09-9612-d3d98102b3a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61b61188b5b94a51ba1ae1a88b78b89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:47:36.579415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d67cf5-0ae1-4e09-9612-d3d98102b3a1 service swift] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:47:36.579694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d67cf5-0ae1-4e09-9612-d3d98102b3a1 service swift] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:47:36.579965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d67cf5-0ae1-4e09-9612-d3d98102b3a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:47:36.588297 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d67cf5-0ae1-4e09-9612-d3d98102b3a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:47:36.589895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d67cf5-0ae1-4e09-9612-d3d98102b3a1 service swift] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:47:36.598460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1262/2517] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jul 21 05:47:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 05:47:46.534007 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:47:46.594974 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=53 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:48:14.963276 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c931c5f-6a94-4841-bdfb-7ccbb685742f None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:14.963855 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c931c5f-6a94-4841-bdfb-7ccbb685742f None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:14.963855 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c931c5f-6a94-4841-bdfb-7ccbb685742f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:14.982621 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-8c931c5f-6a94-4841-bdfb-7ccbb685742f None None] MFA Rules not processed for user `4d04998b636649d98103b1cd5f3a0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:48:14.983848 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8c931c5f-6a94-4841-bdfb-7ccbb685742f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:48:14.992393 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1256/2518] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jul 21 05:48:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:48:15.005310 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d67cf5-0ae1-4e09-9612-d3d98102b3a1 service swift] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:48:15.016497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d67cf5-0ae1-4e09-9612-d3d98102b3a1 service swift] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:48:15.022224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba836e4-6cad-4e16-9f3f-d0c3ad07e1a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61b61188b5b94a51ba1ae1a88b78b89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:48:15.022872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba836e4-6cad-4e16-9f3f-d0c3ad07e1a7 service swift] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:15.023189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba836e4-6cad-4e16-9f3f-d0c3ad07e1a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:15.023450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba836e4-6cad-4e16-9f3f-d0c3ad07e1a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:15.032894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba836e4-6cad-4e16-9f3f-d0c3ad07e1a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:48:15.034825 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba836e4-6cad-4e16-9f3f-d0c3ad07e1a7 service swift] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:48:15.044271 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1263/2519] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jul 21 05:48:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 05:48:20.036834 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65285984-d09f-4c48-ab63-479d58aa0605 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:20.037799 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65285984-d09f-4c48-ab63-479d58aa0605 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:20.038443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65285984-d09f-4c48-ab63-479d58aa0605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:20.072918 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-65285984-d09f-4c48-ab63-479d58aa0605 None None] MFA Rules not processed for user `5baede5b70674737ad285b5831e11002`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:48:20.074907 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-65285984-d09f-4c48-ab63-479d58aa0605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:48:20.159128 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1257/2520] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:48:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:48:21.561545 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba836e4-6cad-4e16-9f3f-d0c3ad07e1a7 service swift] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:48:21.673185 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba836e4-6cad-4e16-9f3f-d0c3ad07e1a7 service swift] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:48:21.678464 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689deda-0153-4d46-8c11-bad1becbdf5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:48:21.679774 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689deda-0153-4d46-8c11-bad1becbdf5a service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:21.680229 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689deda-0153-4d46-8c11-bad1becbdf5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:21.680567 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689deda-0153-4d46-8c11-bad1becbdf5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:21.691125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689deda-0153-4d46-8c11-bad1becbdf5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:48:21.692978 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689deda-0153-4d46-8c11-bad1becbdf5a service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:48:21.704513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1264/2521] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:48:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:48:30.144645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=28 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:48:31.699051 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=32 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:48:33.294187 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905614c-e029-4cd6-9ecd-8f269463c2bb None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:33.295139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905614c-e029-4cd6-9ecd-8f269463c2bb None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:33.295420 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905614c-e029-4cd6-9ecd-8f269463c2bb None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:33.297035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1258/2522] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jul 21 05:48:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:48:33.303518 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43acd1-00d2-4bd9-9f5a-6cb3b0afa663 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:33.304017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43acd1-00d2-4bd9-9f5a-6cb3b0afa663 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:33.304157 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43acd1-00d2-4bd9-9f5a-6cb3b0afa663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:33.327299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-bb43acd1-00d2-4bd9-9f5a-6cb3b0afa663 None None] MFA Rules not processed for user `928d014648cd41d7b58e7e01845eb748`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:48:33.328612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-bb43acd1-00d2-4bd9-9f5a-6cb3b0afa663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:48:33.400566 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1265/2523] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jul 21 05:48:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:48:33.412942 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905614c-e029-4cd6-9ecd-8f269463c2bb None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:48:33.423728 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905614c-e029-4cd6-9ecd-8f269463c2bb None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:48:33.427337 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3c77d-a967-4d32-ae36-c7c6abc480cd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2707c6967a0a419a98ae142ff1d1a961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:48:33.428289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3c77d-a967-4d32-ae36-c7c6abc480cd service glance] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:33.428638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3c77d-a967-4d32-ae36-c7c6abc480cd service glance] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:33.428975 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3c77d-a967-4d32-ae36-c7c6abc480cd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:33.440699 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3c77d-a967-4d32-ae36-c7c6abc480cd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:48:33.442408 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3c77d-a967-4d32-ae36-c7c6abc480cd service glance] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:48:33.457603 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1259/2524] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:48:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:48:33.530450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ead69-e6ca-43da-825a-bcfac6d62ca6 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:33.530775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ead69-e6ca-43da-825a-bcfac6d62ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:33.531103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ead69-e6ca-43da-825a-bcfac6d62ca6 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:33.532500 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1266/2525] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jul 21 05:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:48:33.537910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5d631-cef9-412c-a2c3-0cae7a4cdf71 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:33.538201 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5d631-cef9-412c-a2c3-0cae7a4cdf71 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:33.538477 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5d631-cef9-412c-a2c3-0cae7a4cdf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:33.558741 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-d7b5d631-cef9-412c-a2c3-0cae7a4cdf71 None None] MFA Rules not processed for user `e07508a9fa9146528620b7f8abec8549`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:48:33.560174 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d7b5d631-cef9-412c-a2c3-0cae7a4cdf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:48:33.567663 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1260/2526] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jul 21 05:48:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:48:33.580402 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ead69-e6ca-43da-825a-bcfac6d62ca6 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:48:33.592996 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ead69-e6ca-43da-825a-bcfac6d62ca6 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:48:33.597452 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c92f8-232c-4e9f-925a-5770db710a28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:48:33.598184 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c92f8-232c-4e9f-925a-5770db710a28 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:33.598462 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c92f8-232c-4e9f-925a-5770db710a28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:33.598732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c92f8-232c-4e9f-925a-5770db710a28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:33.606723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c92f8-232c-4e9f-925a-5770db710a28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:48:33.608240 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c92f8-232c-4e9f-925a-5770db710a28 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:48:33.616331 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1267/2527] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:48:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:48:33.703896 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d494c2-f521-4df0-88de-a7d8cd5f791b None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:33.704187 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d494c2-f521-4df0-88de-a7d8cd5f791b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:33.704461 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d494c2-f521-4df0-88de-a7d8cd5f791b None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:33.705935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1261/2528] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jul 21 05:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:48:33.711507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87addbf2-b0cf-450c-80e6-b8d6b5e10fc0 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:33.711799 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87addbf2-b0cf-450c-80e6-b8d6b5e10fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:33.712077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87addbf2-b0cf-450c-80e6-b8d6b5e10fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:33.732101 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-87addbf2-b0cf-450c-80e6-b8d6b5e10fc0 None None] MFA Rules not processed for user `928d014648cd41d7b58e7e01845eb748`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:48:33.733311 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-87addbf2-b0cf-450c-80e6-b8d6b5e10fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:48:33.744215 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1268/2529] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jul 21 05:48:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:48:33.755575 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d494c2-f521-4df0-88de-a7d8cd5f791b None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:48:33.767914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d494c2-f521-4df0-88de-a7d8cd5f791b None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:48:33.771959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3977c-408d-43c3-a744-de9bae6d1efb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:48:33.772599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3977c-408d-43c3-a744-de9bae6d1efb service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:33.772884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3977c-408d-43c3-a744-de9bae6d1efb service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:33.773162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3977c-408d-43c3-a744-de9bae6d1efb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:33.784035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3977c-408d-43c3-a744-de9bae6d1efb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:48:33.785521 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3977c-408d-43c3-a744-de9bae6d1efb service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:48:33.796516 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1262/2530] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:48:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:48:37.024077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9ecb1-e38d-4cc4-abcd-65cd1fcfc0a7 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:37.024562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9ecb1-e38d-4cc4-abcd-65cd1fcfc0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:37.024668 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9ecb1-e38d-4cc4-abcd-65cd1fcfc0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:37.043948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-a0c9ecb1-e38d-4cc4-abcd-65cd1fcfc0a7 None None] MFA Rules not processed for user `4d04998b636649d98103b1cd5f3a0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:48:37.045078 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a0c9ecb1-e38d-4cc4-abcd-65cd1fcfc0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:48:37.052822 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1269/2531] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jul 21 05:48:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:48:37.067508 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3977c-408d-43c3-a744-de9bae6d1efb service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:48:37.078808 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3977c-408d-43c3-a744-de9bae6d1efb service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:48:37.083882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca0036-71b9-4141-8d47-33104fb09bc3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61b61188b5b94a51ba1ae1a88b78b89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:48:37.084529 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca0036-71b9-4141-8d47-33104fb09bc3 service swift] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:37.084815 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca0036-71b9-4141-8d47-33104fb09bc3 service swift] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:37.085092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca0036-71b9-4141-8d47-33104fb09bc3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:37.093656 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca0036-71b9-4141-8d47-33104fb09bc3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:48:37.095274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca0036-71b9-4141-8d47-33104fb09bc3 service swift] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:48:37.103822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1263/2532] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jul 21 05:48:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 05:48:39.669334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9ecb1-e38d-4cc4-abcd-65cd1fcfc0a7 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:48:39.679681 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9ecb1-e38d-4cc4-abcd-65cd1fcfc0a7 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:48:39.683897 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa70fd-dc1c-4aaa-b132-20f3d5e50c7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:48:39.684535 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa70fd-dc1c-4aaa-b132-20f3d5e50c7a service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:39.684816 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa70fd-dc1c-4aaa-b132-20f3d5e50c7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:39.685087 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa70fd-dc1c-4aaa-b132-20f3d5e50c7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:39.687107 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8daa70fd-dc1c-4aaa-b132-20f3d5e50c7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:48:39.701419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa70fd-dc1c-4aaa-b132-20f3d5e50c7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:48:39.703152 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa70fd-dc1c-4aaa-b132-20f3d5e50c7a service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:48:39.716616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1270/2533] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:48:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:48:40.907480 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca0036-71b9-4141-8d47-33104fb09bc3 service swift] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:48:40.918692 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca0036-71b9-4141-8d47-33104fb09bc3 service swift] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:48:40.928340 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec5a7e-9708-42bf-abae-aecb1818642e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:48:40.929006 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec5a7e-9708-42bf-abae-aecb1818642e service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:40.929288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec5a7e-9708-42bf-abae-aecb1818642e service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:40.929559 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec5a7e-9708-42bf-abae-aecb1818642e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:40.931930 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-dcec5a7e-9708-42bf-abae-aecb1818642e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:48:40.946965 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec5a7e-9708-42bf-abae-aecb1818642e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:48:40.948440 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec5a7e-9708-42bf-abae-aecb1818642e service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:48:40.960789 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1264/2534] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:48:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:48:45.656882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa70fd-dc1c-4aaa-b132-20f3d5e50c7a service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:48:45.656882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8daa70fd-dc1c-4aaa-b132-20f3d5e50c7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:48:45.673135 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa70fd-dc1c-4aaa-b132-20f3d5e50c7a service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:48:45.681690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6ad9b-e74f-4146-a68c-8f63bc594c19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:48:45.682344 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6ad9b-e74f-4146-a68c-8f63bc594c19 service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:45.682686 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6ad9b-e74f-4146-a68c-8f63bc594c19 service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:45.682874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6ad9b-e74f-4146-a68c-8f63bc594c19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:45.685091 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-73a6ad9b-e74f-4146-a68c-8f63bc594c19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:48:45.749175 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6ad9b-e74f-4146-a68c-8f63bc594c19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:48:45.750826 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6ad9b-e74f-4146-a68c-8f63bc594c19 service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:48:45.761103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1271/2535] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:48:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:48:50.956363 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=27 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:48:55.756366 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=77 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:48:59.833786 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec5a7e-9708-42bf-abae-aecb1818642e service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:48:59.847830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec5a7e-9708-42bf-abae-aecb1818642e service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:48:59.854982 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc7506-9237-4fc2-8fba-4d1da1273856 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:48:59.855668 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc7506-9237-4fc2-8fba-4d1da1273856 service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:48:59.855951 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc7506-9237-4fc2-8fba-4d1da1273856 service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:48:59.856229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc7506-9237-4fc2-8fba-4d1da1273856 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:48:59.858393 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a8dc7506-9237-4fc2-8fba-4d1da1273856 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:48:59.921631 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc7506-9237-4fc2-8fba-4d1da1273856 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:48:59.923423 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dc7506-9237-4fc2-8fba-4d1da1273856 service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:48:59.933694 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1265/2536] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:48:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:49:09.928191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=28 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:49:11.456851 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6ad9b-e74f-4146-a68c-8f63bc594c19 service nova] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:11.468364 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6ad9b-e74f-4146-a68c-8f63bc594c19 service nova] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:11.472658 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3be3d0-94af-4713-ba91-f55422ffd8fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:11.473349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3be3d0-94af-4713-ba91-f55422ffd8fe service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:11.473632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3be3d0-94af-4713-ba91-f55422ffd8fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:11.473909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3be3d0-94af-4713-ba91-f55422ffd8fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:11.475858 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-3d3be3d0-94af-4713-ba91-f55422ffd8fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:49:11.486039 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3be3d0-94af-4713-ba91-f55422ffd8fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:11.487609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3be3d0-94af-4713-ba91-f55422ffd8fe service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:11.496009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1272/2537] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:49:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:49:15.656841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607924e5-2d99-4990-aca7-4154e7df301b None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:15.657344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607924e5-2d99-4990-aca7-4154e7df301b None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:15.657344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607924e5-2d99-4990-aca7-4154e7df301b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:15.676607 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-607924e5-2d99-4990-aca7-4154e7df301b None None] MFA Rules not processed for user `4d04998b636649d98103b1cd5f3a0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:49:15.677838 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-607924e5-2d99-4990-aca7-4154e7df301b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:49:15.686739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1266/2538] 10.222.0.44 () {58 vars in 967 bytes} [Thu Jul 21 05:49:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:49:15.701490 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3be3d0-94af-4713-ba91-f55422ffd8fe service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:15.715394 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3be3d0-94af-4713-ba91-f55422ffd8fe service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:15.721278 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2244c9-72b2-4053-bc0c-f1a175c575b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61b61188b5b94a51ba1ae1a88b78b89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:15.722024 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2244c9-72b2-4053-bc0c-f1a175c575b0 service swift] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:15.722383 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2244c9-72b2-4053-bc0c-f1a175c575b0 service swift] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:15.722669 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2244c9-72b2-4053-bc0c-f1a175c575b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:15.731233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2244c9-72b2-4053-bc0c-f1a175c575b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:15.732835 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2244c9-72b2-4053-bc0c-f1a175c575b0 service swift] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:15.741059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1273/2539] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jul 21 05:49:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 05:49:21.375882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607924e5-2d99-4990-aca7-4154e7df301b None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:21.389482 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607924e5-2d99-4990-aca7-4154e7df301b None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:21.395303 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb2d31-2225-4f51-854f-52f7294eef0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:21.396036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb2d31-2225-4f51-854f-52f7294eef0c service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:21.396443 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb2d31-2225-4f51-854f-52f7294eef0c service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:21.397058 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb2d31-2225-4f51-854f-52f7294eef0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:21.399630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7cfb2d31-2225-4f51-854f-52f7294eef0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:49:21.417593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb2d31-2225-4f51-854f-52f7294eef0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:21.419315 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb2d31-2225-4f51-854f-52f7294eef0c service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:21.431228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1267/2540] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:49:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:49:25.737797 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=12 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:49:31.426887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=9 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:49:40.981181 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a829eb5-1d21-4051-85f0-507ad856da33 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:40.981716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a829eb5-1d21-4051-85f0-507ad856da33 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:40.981716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a829eb5-1d21-4051-85f0-507ad856da33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:41.000860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-7a829eb5-1d21-4051-85f0-507ad856da33 None None] MFA Rules not processed for user `563f1367093e493bae87a21374a3c2f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:49:41.002093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-7a829eb5-1d21-4051-85f0-507ad856da33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:49:41.010116 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1274/2541] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:49:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:49:48.191662 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb2d31-2225-4f51-854f-52f7294eef0c service nova] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:48.193755 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7cfb2d31-2225-4f51-854f-52f7294eef0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:49:48.267816 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb2d31-2225-4f51-854f-52f7294eef0c service nova] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:48.272778 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eacb31-277d-4fd4-bcc4-9493e569d917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:48.273660 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eacb31-277d-4fd4-bcc4-9493e569d917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:48.273944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eacb31-277d-4fd4-bcc4-9493e569d917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:48.274220 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eacb31-277d-4fd4-bcc4-9493e569d917 admin admin] PATH_INFO: `/v3/users/5baede5b70674737ad285b5831e11002` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:48.275574 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eacb31-277d-4fd4-bcc4-9493e569d917 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5baede5b70674737ad285b5831e11002)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:48.277526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eacb31-277d-4fd4-bcc4-9493e569d917 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:48.327619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-a1eacb31-277d-4fd4-bcc4-9493e569d917 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5baede5b70674737ad285b5831e11002'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:49:48.338883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-a1eacb31-277d-4fd4-bcc4-9493e569d917 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5baede5b70674737ad285b5831e11002'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:49:48.345923 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-a1eacb31-277d-4fd4-bcc4-9493e569d917 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5baede5b70674737ad285b5831e11002'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:49:48.346553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-a1eacb31-277d-4fd4-bcc4-9493e569d917 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5baede5b70674737ad285b5831e11002'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:49:48.347181 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-a1eacb31-277d-4fd4-bcc4-9493e569d917 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5baede5b70674737ad285b5831e11002'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:49:48.361194 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1268/2542] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:49:48 2022] DELETE /identity/v3/users/5baede5b70674737ad285b5831e11002 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:49:48.550718 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a829eb5-1d21-4051-85f0-507ad856da33 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:48.564903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a829eb5-1d21-4051-85f0-507ad856da33 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:48.629962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cef00d-92c1-424a-af9e-5af1160a0e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:48.630837 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cef00d-92c1-424a-af9e-5af1160a0e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:48.631161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cef00d-92c1-424a-af9e-5af1160a0e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:48.631436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cef00d-92c1-424a-af9e-5af1160a0e0d admin admin] PATH_INFO: `/v3/projects/cb4a02ef918f418f9509f0698503f428` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:48.632774 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cef00d-92c1-424a-af9e-5af1160a0e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb4a02ef918f418f9509f0698503f428)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:48.634673 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cef00d-92c1-424a-af9e-5af1160a0e0d admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:48.684268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b1cef00d-92c1-424a-af9e-5af1160a0e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb4a02ef918f418f9509f0698503f428', 'request_id': 'req-b1cef00d-92c1-424a-af9e-5af1160a0e0d'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:49:48.690082 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b1cef00d-92c1-424a-af9e-5af1160a0e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb4a02ef918f418f9509f0698503f428', 'request_id': 'req-b1cef00d-92c1-424a-af9e-5af1160a0e0d'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:49:48.706531 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b1cef00d-92c1-424a-af9e-5af1160a0e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb4a02ef918f418f9509f0698503f428', 'request_id': 'req-b1cef00d-92c1-424a-af9e-5af1160a0e0d'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:49:48.725994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b1cef00d-92c1-424a-af9e-5af1160a0e0d admin admin] The token cache is being invalidate because project cb4a02ef918f418f9509f0698503f428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:49:48.726252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b1cef00d-92c1-424a-af9e-5af1160a0e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:49:48.727362 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b1cef00d-92c1-424a-af9e-5af1160a0e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:49:48.729947 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1275/2543] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:49:48 2022] DELETE /identity/v3/projects/cb4a02ef918f418f9509f0698503f428 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:49:48.738153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb493ea9-5dd3-4044-a65d-0b796303467f None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:48.738514 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb493ea9-5dd3-4044-a65d-0b796303467f None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:48.738903 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb493ea9-5dd3-4044-a65d-0b796303467f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:48.767407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-cb493ea9-5dd3-4044-a65d-0b796303467f None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:49:48.768641 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-cb493ea9-5dd3-4044-a65d-0b796303467f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:49:48.831760 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1269/2544] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:49:48.840409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a53de-0601-49e7-b57d-0f04a8a88ffb None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:48.840709 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a53de-0601-49e7-b57d-0f04a8a88ffb None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:48.840989 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a53de-0601-49e7-b57d-0f04a8a88ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:48.860418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-aa2a53de-0601-49e7-b57d-0f04a8a88ffb None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:49:48.861635 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-aa2a53de-0601-49e7-b57d-0f04a8a88ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:49:48.870113 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1276/2545] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:49:48.874611 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb493ea9-5dd3-4044-a65d-0b796303467f None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:48.884790 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb493ea9-5dd3-4044-a65d-0b796303467f None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:48.890434 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0caa4b-33c4-4423-8cfd-0af56c1db1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:48.891247 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0caa4b-33c4-4423-8cfd-0af56c1db1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:48.891535 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0caa4b-33c4-4423-8cfd-0af56c1db1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:48.891817 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0caa4b-33c4-4423-8cfd-0af56c1db1e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:48.892549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0caa4b-33c4-4423-8cfd-0af56c1db1e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:49:48.892856 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0caa4b-33c4-4423-8cfd-0af56c1db1e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:48.894563 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0caa4b-33c4-4423-8cfd-0af56c1db1e1 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:48.908017 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1270/2546] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:49:48.912939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a53de-0601-49e7-b57d-0f04a8a88ffb None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:48.924882 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a53de-0601-49e7-b57d-0f04a8a88ffb None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:48.930527 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab920b-6059-4d9a-bcdb-7307feee873a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:48.931213 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab920b-6059-4d9a-bcdb-7307feee873a admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:48.931497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab920b-6059-4d9a-bcdb-7307feee873a admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:48.931770 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab920b-6059-4d9a-bcdb-7307feee873a admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:48.932473 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab920b-6059-4d9a-bcdb-7307feee873a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:48.934396 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ab920b-6059-4d9a-bcdb-7307feee873a admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:48.962272 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1277/2547] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:49:48 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:49:48.967194 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0caa4b-33c4-4423-8cfd-0af56c1db1e1 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:48.980315 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0caa4b-33c4-4423-8cfd-0af56c1db1e1 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:48.982427 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:49:49.037138 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b508a-37d7-4002-9f76-b20cf375bf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:49.037851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b508a-37d7-4002-9f76-b20cf375bf6b admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:49.038133 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b508a-37d7-4002-9f76-b20cf375bf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:49.038407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b508a-37d7-4002-9f76-b20cf375bf6b admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:49.039120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b508a-37d7-4002-9f76-b20cf375bf6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:49.041038 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b508a-37d7-4002-9f76-b20cf375bf6b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:49.063058 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1271/2548] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:49:48 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:49:49.067389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab920b-6059-4d9a-bcdb-7307feee873a admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:49.078319 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab920b-6059-4d9a-bcdb-7307feee873a admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:49.082952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb94f9c-4978-4a4b-b142-f503e7ef92fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:49.083646 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb94f9c-4978-4a4b-b142-f503e7ef92fd admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:49.083925 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb94f9c-4978-4a4b-b142-f503e7ef92fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:49.084197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb94f9c-4978-4a4b-b142-f503e7ef92fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:49.084924 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb94f9c-4978-4a4b-b142-f503e7ef92fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:49.086533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb94f9c-4978-4a4b-b142-f503e7ef92fd admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:49.099312 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1278/2549] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:49:49.104063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b508a-37d7-4002-9f76-b20cf375bf6b admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:49.114581 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b508a-37d7-4002-9f76-b20cf375bf6b admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:49.120176 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475f5e4-4ea7-4802-a584-b14c83799298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:49.121160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475f5e4-4ea7-4802-a584-b14c83799298 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:49.121160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475f5e4-4ea7-4802-a584-b14c83799298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:49.121403 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475f5e4-4ea7-4802-a584-b14c83799298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:49.122116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475f5e4-4ea7-4802-a584-b14c83799298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:49.123780 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475f5e4-4ea7-4802-a584-b14c83799298 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:49.134503 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1272/2550] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:49:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:49:49.138787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb94f9c-4978-4a4b-b142-f503e7ef92fd admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:49.149331 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb94f9c-4978-4a4b-b142-f503e7ef92fd admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:49.154887 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433a183-d071-4997-afaa-2579eaac15a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:49.155813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433a183-d071-4997-afaa-2579eaac15a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:49.156080 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433a183-d071-4997-afaa-2579eaac15a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:49.156338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433a183-d071-4997-afaa-2579eaac15a2 admin admin] PATH_INFO: `/v3/projects/0c6f9207778b4e0fbdbbae28c8f7b224/users/17108265cfba44b29c362ac18bbf3e20/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:49.173556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433a183-d071-4997-afaa-2579eaac15a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c6f9207778b4e0fbdbbae28c8f7b224, user_id=17108265cfba44b29c362ac18bbf3e20, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:49.176329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433a183-d071-4997-afaa-2579eaac15a2 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:49.194043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1279/2551] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:49:49 2022] PUT /identity/v3/projects/0c6f9207778b4e0fbdbbae28c8f7b224/users/17108265cfba44b29c362ac18bbf3e20/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:49:49.200765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fed2f-8fda-4e6b-a97c-5a9512512690 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:49.201289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fed2f-8fda-4e6b-a97c-5a9512512690 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:49.201435 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4fed2f-8fda-4e6b-a97c-5a9512512690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:49.222400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-2b4fed2f-8fda-4e6b-a97c-5a9512512690 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:49:49.223205 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-2b4fed2f-8fda-4e6b-a97c-5a9512512690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:49:49.285440 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1273/2552] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:49:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:49:49.295448 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433a183-d071-4997-afaa-2579eaac15a2 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:49.297513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5433a183-d071-4997-afaa-2579eaac15a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:49:49.321114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:49:49.360857 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433a183-d071-4997-afaa-2579eaac15a2 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:49.364977 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71510ddc-5084-4e13-84fe-f7ad0c70020b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:49.365669 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71510ddc-5084-4e13-84fe-f7ad0c70020b service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:49.365957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71510ddc-5084-4e13-84fe-f7ad0c70020b service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:49.366222 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71510ddc-5084-4e13-84fe-f7ad0c70020b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:49.375808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71510ddc-5084-4e13-84fe-f7ad0c70020b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:49.377288 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71510ddc-5084-4e13-84fe-f7ad0c70020b service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:49.387084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1280/2553] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:49:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:49:53.874851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2184987-f68f-48a6-85b9-9f85c910bbf6 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:53.875623 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2184987-f68f-48a6-85b9-9f85c910bbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:53.878522 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2184987-f68f-48a6-85b9-9f85c910bbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:53.913612 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-a2184987-f68f-48a6-85b9-9f85c910bbf6 None None] MFA Rules not processed for user `17108265cfba44b29c362ac18bbf3e20`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:49:53.914974 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a2184987-f68f-48a6-85b9-9f85c910bbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:49:53.996351 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1274/2554] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:49:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:49:54.000880 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71510ddc-5084-4e13-84fe-f7ad0c70020b service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:54.011461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71510ddc-5084-4e13-84fe-f7ad0c70020b service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:54.016550 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da624f-9308-4235-a509-734a112d49e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:54.017516 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da624f-9308-4235-a509-734a112d49e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:54.017903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da624f-9308-4235-a509-734a112d49e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:54.018284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da624f-9308-4235-a509-734a112d49e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:54.019118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da624f-9308-4235-a509-734a112d49e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:54.021162 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da624f-9308-4235-a509-734a112d49e5 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:54.054072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1281/2555] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:49:53 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:49:54.058325 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2184987-f68f-48a6-85b9-9f85c910bbf6 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:54.070006 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2184987-f68f-48a6-85b9-9f85c910bbf6 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:54.086117 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:49:54.150096 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bf971-e1df-4064-84da-479d063e9096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:54.150822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bf971-e1df-4064-84da-479d063e9096 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:54.151160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bf971-e1df-4064-84da-479d063e9096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:54.151441 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bf971-e1df-4064-84da-479d063e9096 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:54.152143 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3bf971-e1df-4064-84da-479d063e9096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:54.154097 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3bf971-e1df-4064-84da-479d063e9096 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:54.175400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1275/2556] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:49:54 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:49:54.179629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da624f-9308-4235-a509-734a112d49e5 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:54.189902 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da624f-9308-4235-a509-734a112d49e5 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:54.194475 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50473ecf-ad49-45ef-998b-1dd41c334adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:54.195250 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50473ecf-ad49-45ef-998b-1dd41c334adb admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:54.195604 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50473ecf-ad49-45ef-998b-1dd41c334adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:54.195953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50473ecf-ad49-45ef-998b-1dd41c334adb admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:54.196787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50473ecf-ad49-45ef-998b-1dd41c334adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:54.198456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50473ecf-ad49-45ef-998b-1dd41c334adb admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:54.209449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1282/2557] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:49:54.214281 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bf971-e1df-4064-84da-479d063e9096 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:54.224976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bf971-e1df-4064-84da-479d063e9096 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:54.230101 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96099c8f-7203-4621-9230-fd5a7410d7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:54.231130 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96099c8f-7203-4621-9230-fd5a7410d7a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:54.231419 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96099c8f-7203-4621-9230-fd5a7410d7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:54.231699 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96099c8f-7203-4621-9230-fd5a7410d7a4 admin admin] PATH_INFO: `/v3/projects/de984aea424f468e8eb9d92d44ccbad2/users/5d08b387d141426eb831137934589a2e/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:54.248108 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96099c8f-7203-4621-9230-fd5a7410d7a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de984aea424f468e8eb9d92d44ccbad2, user_id=5d08b387d141426eb831137934589a2e, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:54.250867 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96099c8f-7203-4621-9230-fd5a7410d7a4 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:54.264514 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1276/2558] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:49:54 2022] PUT /identity/v3/projects/de984aea424f468e8eb9d92d44ccbad2/users/5d08b387d141426eb831137934589a2e/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:49:54.269030 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50473ecf-ad49-45ef-998b-1dd41c334adb admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:54.279639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50473ecf-ad49-45ef-998b-1dd41c334adb admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:54.347335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1adb4c-773f-47dd-9cce-d16079df3c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:54.348019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1adb4c-773f-47dd-9cce-d16079df3c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:54.348306 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1adb4c-773f-47dd-9cce-d16079df3c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:54.348585 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1adb4c-773f-47dd-9cce-d16079df3c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:54.349327 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1adb4c-773f-47dd-9cce-d16079df3c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:54.350942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1adb4c-773f-47dd-9cce-d16079df3c7b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:54.360973 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1283/2559] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 05:49:54.365324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96099c8f-7203-4621-9230-fd5a7410d7a4 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:54.375966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96099c8f-7203-4621-9230-fd5a7410d7a4 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:54.380851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c1ede-e14a-4edf-bebd-e68582b10f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:54.381749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c1ede-e14a-4edf-bebd-e68582b10f7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:54.382031 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c1ede-e14a-4edf-bebd-e68582b10f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:54.382303 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c1ede-e14a-4edf-bebd-e68582b10f7d admin admin] PATH_INFO: `/v3/projects/de984aea424f468e8eb9d92d44ccbad2/users/5d08b387d141426eb831137934589a2e/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:54.384317 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c1ede-e14a-4edf-bebd-e68582b10f7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=de984aea424f468e8eb9d92d44ccbad2, user_id=5d08b387d141426eb831137934589a2e, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:54.386972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c1ede-e14a-4edf-bebd-e68582b10f7d admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:54.397557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1277/2560] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:49:54 2022] PUT /identity/v3/projects/de984aea424f468e8eb9d92d44ccbad2/users/5d08b387d141426eb831137934589a2e/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:49:58.745069 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ed16c-ffb1-4142-883c-e35103641f4a None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:58.745914 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ed16c-ffb1-4142-883c-e35103641f4a None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:58.746197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ed16c-ffb1-4142-883c-e35103641f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:58.786773 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-4e5ed16c-ffb1-4142-883c-e35103641f4a None None] MFA Rules not processed for user `5d08b387d141426eb831137934589a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:49:58.788069 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4e5ed16c-ffb1-4142-883c-e35103641f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:49:58.875265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1284/2561] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:49:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:49:58.893594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c1ede-e14a-4edf-bebd-e68582b10f7d admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:49:58.896284 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-507c1ede-e14a-4edf-bebd-e68582b10f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:49:58.985947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c1ede-e14a-4edf-bebd-e68582b10f7d admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:49:58.991982 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198c8c4-8797-4de5-a106-80f18082beba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:49:58.992677 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198c8c4-8797-4de5-a106-80f18082beba service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:49:58.992964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198c8c4-8797-4de5-a106-80f18082beba service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:49:58.993242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198c8c4-8797-4de5-a106-80f18082beba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:49:59.056667 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198c8c4-8797-4de5-a106-80f18082beba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:49:59.058319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198c8c4-8797-4de5-a106-80f18082beba service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:49:59.068720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1278/2562] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:49:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:50:08.871299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:50:09.064743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:50:24.833843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fa0bc-205d-4b3b-af36-40c53248cdb9 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:50:24.834412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fa0bc-205d-4b3b-af36-40c53248cdb9 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:50:24.834412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fa0bc-205d-4b3b-af36-40c53248cdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:50:24.853573 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-b33fa0bc-205d-4b3b-af36-40c53248cdb9 None None] MFA Rules not processed for user `17108265cfba44b29c362ac18bbf3e20`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:50:24.854915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b33fa0bc-205d-4b3b-af36-40c53248cdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:50:24.863638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1285/2563] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:50:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:50:25.201601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198c8c4-8797-4de5-a106-80f18082beba service nova] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:50:25.203229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-e198c8c4-8797-4de5-a106-80f18082beba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:50:25.262774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198c8c4-8797-4de5-a106-80f18082beba service nova] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:50:25.267685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee621c-1ab5-49d5-b2cb-c07314c0c8ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:50:25.269480 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee621c-1ab5-49d5-b2cb-c07314c0c8ba service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:50:25.269793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee621c-1ab5-49d5-b2cb-c07314c0c8ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:50:25.270071 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee621c-1ab5-49d5-b2cb-c07314c0c8ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:50:25.272834 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-afee621c-1ab5-49d5-b2cb-c07314c0c8ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:50:25.286772 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee621c-1ab5-49d5-b2cb-c07314c0c8ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:50:25.288300 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee621c-1ab5-49d5-b2cb-c07314c0c8ba service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:50:25.300742 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1279/2564] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:50:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:50:25.806160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33fa0bc-205d-4b3b-af36-40c53248cdb9 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:50:25.808177 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b33fa0bc-205d-4b3b-af36-40c53248cdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:50:25.882002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33fa0bc-205d-4b3b-af36-40c53248cdb9 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:50:25.891629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6ed64-7f11-4fbe-ad5f-5ac2ea7d7ee1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:50:25.892286 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6ed64-7f11-4fbe-ad5f-5ac2ea7d7ee1 service placement] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:50:25.892563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6ed64-7f11-4fbe-ad5f-5ac2ea7d7ee1 service placement] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:50:25.892830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6ed64-7f11-4fbe-ad5f-5ac2ea7d7ee1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:50:25.894696 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-cfe6ed64-7f11-4fbe-ad5f-5ac2ea7d7ee1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:50:25.906238 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6ed64-7f11-4fbe-ad5f-5ac2ea7d7ee1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:50:25.908292 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6ed64-7f11-4fbe-ad5f-5ac2ea7d7ee1 service placement] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:50:25.918438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1286/2565] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:50:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:50:35.296542 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=26 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:50:35.913927 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=35 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:50:41.812567 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee621c-1ab5-49d5-b2cb-c07314c0c8ba service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:50:41.825445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee621c-1ab5-49d5-b2cb-c07314c0c8ba service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:50:41.831507 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94659c-c450-43dd-9a83-688ce28fecc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:50:41.832194 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94659c-c450-43dd-9a83-688ce28fecc1 service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:50:41.832488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94659c-c450-43dd-9a83-688ce28fecc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:50:41.833077 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94659c-c450-43dd-9a83-688ce28fecc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:50:41.834675 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-cf94659c-c450-43dd-9a83-688ce28fecc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:50:41.849100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94659c-c450-43dd-9a83-688ce28fecc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:50:41.850743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94659c-c450-43dd-9a83-688ce28fecc1 service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:50:41.862009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1280/2566] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:50:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:50:45.543488 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6ed64-7f11-4fbe-ad5f-5ac2ea7d7ee1 service placement] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:50:45.553468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6ed64-7f11-4fbe-ad5f-5ac2ea7d7ee1 service placement] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:50:45.557891 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaba3db-5b55-45fa-b19d-a354210328ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:50:45.558533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaba3db-5b55-45fa-b19d-a354210328ba service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:50:45.559044 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaba3db-5b55-45fa-b19d-a354210328ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:50:45.559361 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaba3db-5b55-45fa-b19d-a354210328ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:50:45.569444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaba3db-5b55-45fa-b19d-a354210328ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:50:45.570892 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaba3db-5b55-45fa-b19d-a354210328ba service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:50:45.580563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1287/2567] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:50:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:50:51.857369 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=6 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:50:55.576328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=6 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:10.205456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94659c-c450-43dd-9a83-688ce28fecc1 service nova] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:10.217910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94659c-c450-43dd-9a83-688ce28fecc1 service nova] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:10.222550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06d644-5a67-44dd-b5ec-d71f1dba9640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:10.223246 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06d644-5a67-44dd-b5ec-d71f1dba9640 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:10.223528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06d644-5a67-44dd-b5ec-d71f1dba9640 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:10.223809 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06d644-5a67-44dd-b5ec-d71f1dba9640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:10.225850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5b06d644-5a67-44dd-b5ec-d71f1dba9640 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:10.291704 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06d644-5a67-44dd-b5ec-d71f1dba9640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:10.293155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06d644-5a67-44dd-b5ec-d71f1dba9640 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:10.303180 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1281/2568] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:51:13.462460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaba3db-5b55-45fa-b19d-a354210328ba service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:13.466363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-edaba3db-5b55-45fa-b19d-a354210328ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:13.489927 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaba3db-5b55-45fa-b19d-a354210328ba service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:13.496236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d4357-8f36-4964-baeb-8bd37158f7b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:13.497169 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d4357-8f36-4964-baeb-8bd37158f7b6 service placement] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:13.497665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d4357-8f36-4964-baeb-8bd37158f7b6 service placement] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:13.498140 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d4357-8f36-4964-baeb-8bd37158f7b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:13.501413 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-cf9d4357-8f36-4964-baeb-8bd37158f7b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:13.516044 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d4357-8f36-4964-baeb-8bd37158f7b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:13.517936 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d4357-8f36-4964-baeb-8bd37158f7b6 service placement] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:13.529458 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1288/2569] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:51:13.539834 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06d644-5a67-44dd-b5ec-d71f1dba9640 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:13.541920 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5b06d644-5a67-44dd-b5ec-d71f1dba9640 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:13.556193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06d644-5a67-44dd-b5ec-d71f1dba9640 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:13.561356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf9597-e108-4b1d-b5ec-d804670466cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:13.562449 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf9597-e108-4b1d-b5ec-d804670466cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:13.562847 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf9597-e108-4b1d-b5ec-d804670466cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:13.563351 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf9597-e108-4b1d-b5ec-d804670466cf admin admin] PATH_INFO: `/v3/users/563f1367093e493bae87a21374a3c2f3` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:13.564842 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf9597-e108-4b1d-b5ec-d804670466cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=563f1367093e493bae87a21374a3c2f3)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:13.567219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf9597-e108-4b1d-b5ec-d804670466cf admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:13.634600 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-14cf9597-e108-4b1d-b5ec-d804670466cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '563f1367093e493bae87a21374a3c2f3'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:13.648560 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-14cf9597-e108-4b1d-b5ec-d804670466cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '563f1367093e493bae87a21374a3c2f3'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:13.656571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-14cf9597-e108-4b1d-b5ec-d804670466cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '563f1367093e493bae87a21374a3c2f3'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:13.658090 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-14cf9597-e108-4b1d-b5ec-d804670466cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '563f1367093e493bae87a21374a3c2f3'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:13.658718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-14cf9597-e108-4b1d-b5ec-d804670466cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '563f1367093e493bae87a21374a3c2f3'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:13.673170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1282/2570] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:51:13 2022] DELETE /identity/v3/users/563f1367093e493bae87a21374a3c2f3 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:13.855899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d4357-8f36-4964-baeb-8bd37158f7b6 service placement] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:13.870957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d4357-8f36-4964-baeb-8bd37158f7b6 service placement] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:13.940638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13337f-26ad-4124-8cd7-79c9cd4b0177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:13.941527 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13337f-26ad-4124-8cd7-79c9cd4b0177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:13.941815 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13337f-26ad-4124-8cd7-79c9cd4b0177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:13.942094 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13337f-26ad-4124-8cd7-79c9cd4b0177 admin admin] PATH_INFO: `/v3/projects/da2259120cca423686fd6e54d0ad2257` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:13.943314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13337f-26ad-4124-8cd7-79c9cd4b0177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da2259120cca423686fd6e54d0ad2257)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:13.945257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13337f-26ad-4124-8cd7-79c9cd4b0177 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:13.992624 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-9c13337f-26ad-4124-8cd7-79c9cd4b0177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da2259120cca423686fd6e54d0ad2257', 'request_id': 'req-9c13337f-26ad-4124-8cd7-79c9cd4b0177'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:13.999121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-9c13337f-26ad-4124-8cd7-79c9cd4b0177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da2259120cca423686fd6e54d0ad2257', 'request_id': 'req-9c13337f-26ad-4124-8cd7-79c9cd4b0177'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:14.007068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-9c13337f-26ad-4124-8cd7-79c9cd4b0177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da2259120cca423686fd6e54d0ad2257', 'request_id': 'req-9c13337f-26ad-4124-8cd7-79c9cd4b0177'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:14.018550 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-9c13337f-26ad-4124-8cd7-79c9cd4b0177 admin admin] The token cache is being invalidate because project da2259120cca423686fd6e54d0ad2257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:51:14.018842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-9c13337f-26ad-4124-8cd7-79c9cd4b0177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:14.019586 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-9c13337f-26ad-4124-8cd7-79c9cd4b0177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:14.021203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1289/2571] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:51:13 2022] DELETE /identity/v3/projects/da2259120cca423686fd6e54d0ad2257 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:14.026844 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df4b38-7258-4eb9-ace7-f0f10b580f5a None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:14.027175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df4b38-7258-4eb9-ace7-f0f10b580f5a None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:14.027460 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df4b38-7258-4eb9-ace7-f0f10b580f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:14.045592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-f0df4b38-7258-4eb9-ace7-f0f10b580f5a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:14.046825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f0df4b38-7258-4eb9-ace7-f0f10b580f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:14.109576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1283/2572] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:14.117863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1889d-d397-417b-b450-f39d7fdf8ade None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:14.118149 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1889d-d397-417b-b450-f39d7fdf8ade None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:14.118449 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1889d-d397-417b-b450-f39d7fdf8ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:14.138143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-a2f1889d-d397-417b-b450-f39d7fdf8ade None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:14.139393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a2f1889d-d397-417b-b450-f39d7fdf8ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:14.149002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1290/2573] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:14.153646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df4b38-7258-4eb9-ace7-f0f10b580f5a None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:14.167758 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df4b38-7258-4eb9-ace7-f0f10b580f5a None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:14.173883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875eeec-81de-4484-a731-eb3be6ac4d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:14.174553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875eeec-81de-4484-a731-eb3be6ac4d71 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:14.174839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875eeec-81de-4484-a731-eb3be6ac4d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:14.175163 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5875eeec-81de-4484-a731-eb3be6ac4d71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:14.175897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875eeec-81de-4484-a731-eb3be6ac4d71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:51:14.176207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875eeec-81de-4484-a731-eb3be6ac4d71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:14.177827 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5875eeec-81de-4484-a731-eb3be6ac4d71 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:14.179321 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:14.191061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1284/2574] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:51:14.195937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1889d-d397-417b-b450-f39d7fdf8ade None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:14.205148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1889d-d397-417b-b450-f39d7fdf8ade None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:14.210021 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30fe3f-1865-4d59-b9da-b3ef7c630fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:14.210651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30fe3f-1865-4d59-b9da-b3ef7c630fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:14.210935 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30fe3f-1865-4d59-b9da-b3ef7c630fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:14.211252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30fe3f-1865-4d59-b9da-b3ef7c630fe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:14.211949 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30fe3f-1865-4d59-b9da-b3ef7c630fe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:14.214019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30fe3f-1865-4d59-b9da-b3ef7c630fe5 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:14.241687 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1291/2575] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:51:14 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:14.245866 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875eeec-81de-4484-a731-eb3be6ac4d71 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:14.255932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5875eeec-81de-4484-a731-eb3be6ac4d71 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:14.314593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4847b22-29dc-4f26-aecd-2ba3fd4af3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:14.315301 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4847b22-29dc-4f26-aecd-2ba3fd4af3d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:14.315591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4847b22-29dc-4f26-aecd-2ba3fd4af3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:14.315873 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4847b22-29dc-4f26-aecd-2ba3fd4af3d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:14.316567 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4847b22-29dc-4f26-aecd-2ba3fd4af3d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:14.318513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4847b22-29dc-4f26-aecd-2ba3fd4af3d0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:14.340891 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1285/2576] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:51:14 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:14.345281 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30fe3f-1865-4d59-b9da-b3ef7c630fe5 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:14.356146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30fe3f-1865-4d59-b9da-b3ef7c630fe5 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:14.361289 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db702f-41ca-41e9-8b8a-276bec33e68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:14.362453 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db702f-41ca-41e9-8b8a-276bec33e68b admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:14.362453 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db702f-41ca-41e9-8b8a-276bec33e68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:14.362688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db702f-41ca-41e9-8b8a-276bec33e68b admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:14.363314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db702f-41ca-41e9-8b8a-276bec33e68b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:14.364932 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db702f-41ca-41e9-8b8a-276bec33e68b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:14.376754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1292/2577] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:14.381038 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4847b22-29dc-4f26-aecd-2ba3fd4af3d0 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:14.391744 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4847b22-29dc-4f26-aecd-2ba3fd4af3d0 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:14.397390 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f7d07-8e96-43f8-8f67-3b8d3aa7ee9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:14.398049 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f7d07-8e96-43f8-8f67-3b8d3aa7ee9d admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:14.398330 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f7d07-8e96-43f8-8f67-3b8d3aa7ee9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:14.398601 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f7d07-8e96-43f8-8f67-3b8d3aa7ee9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:14.399359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f7d07-8e96-43f8-8f67-3b8d3aa7ee9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:14.400969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f7d07-8e96-43f8-8f67-3b8d3aa7ee9d admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:14.411739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1286/2578] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:14.415965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db702f-41ca-41e9-8b8a-276bec33e68b admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:14.426593 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db702f-41ca-41e9-8b8a-276bec33e68b admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:14.432298 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cbb7e-6d52-4311-aa7e-150f6443742c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:14.433180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cbb7e-6d52-4311-aa7e-150f6443742c admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:14.433459 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cbb7e-6d52-4311-aa7e-150f6443742c admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:14.433731 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cbb7e-6d52-4311-aa7e-150f6443742c admin admin] PATH_INFO: `/v3/projects/0b28b45a23e540b597b4d83d3b99ddd4/users/4a4411196cd24dcf9864ff250341d153/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:14.449392 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882cbb7e-6d52-4311-aa7e-150f6443742c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b28b45a23e540b597b4d83d3b99ddd4, user_id=4a4411196cd24dcf9864ff250341d153, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:14.452308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882cbb7e-6d52-4311-aa7e-150f6443742c admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:14.466119 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1293/2579] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:51:14 2022] PUT /identity/v3/projects/0b28b45a23e540b597b4d83d3b99ddd4/users/4a4411196cd24dcf9864ff250341d153/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:14.476314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b7410-1ea2-4e03-98d1-3c2813488667 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:14.476609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b7410-1ea2-4e03-98d1-3c2813488667 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:14.476892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b7410-1ea2-4e03-98d1-3c2813488667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:14.503928 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-978b7410-1ea2-4e03-98d1-3c2813488667 None None] MFA Rules not processed for user `4a4411196cd24dcf9864ff250341d153`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:14.505061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-978b7410-1ea2-4e03-98d1-3c2813488667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:14.573769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1287/2580] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:14.588130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cbb7e-6d52-4311-aa7e-150f6443742c admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:14.590215 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-882cbb7e-6d52-4311-aa7e-150f6443742c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:14.604913 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:14.673382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cbb7e-6d52-4311-aa7e-150f6443742c admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:14.679679 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb062122-e4e8-42f5-bf64-31ead9dc379b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:14.680350 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb062122-e4e8-42f5-bf64-31ead9dc379b service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:14.680633 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb062122-e4e8-42f5-bf64-31ead9dc379b service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:14.680920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb062122-e4e8-42f5-bf64-31ead9dc379b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:14.689316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb062122-e4e8-42f5-bf64-31ead9dc379b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:14.690915 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb062122-e4e8-42f5-bf64-31ead9dc379b service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:14.699982 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1294/2581] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:51:17.416964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b7410-1ea2-4e03-98d1-3c2813488667 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:17.485405 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b7410-1ea2-4e03-98d1-3c2813488667 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:17.491960 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7e9b1-78a8-4a65-92f6-779fdb301182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:17.492839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7e9b1-78a8-4a65-92f6-779fdb301182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:17.493119 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7e9b1-78a8-4a65-92f6-779fdb301182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:17.493426 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7e9b1-78a8-4a65-92f6-779fdb301182 admin admin] PATH_INFO: `/v3/users/4a4411196cd24dcf9864ff250341d153` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:17.494775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7e9b1-78a8-4a65-92f6-779fdb301182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a4411196cd24dcf9864ff250341d153)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:17.496752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7e9b1-78a8-4a65-92f6-779fdb301182 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:17.499234 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:17.550226 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b4b7e9b1-78a8-4a65-92f6-779fdb301182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a4411196cd24dcf9864ff250341d153'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:17.561613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b4b7e9b1-78a8-4a65-92f6-779fdb301182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a4411196cd24dcf9864ff250341d153'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:17.568876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b4b7e9b1-78a8-4a65-92f6-779fdb301182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a4411196cd24dcf9864ff250341d153'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:17.569622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b4b7e9b1-78a8-4a65-92f6-779fdb301182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a4411196cd24dcf9864ff250341d153'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:17.570300 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-b4b7e9b1-78a8-4a65-92f6-779fdb301182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a4411196cd24dcf9864ff250341d153'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:17.583828 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1288/2582] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:51:17 2022] DELETE /identity/v3/users/4a4411196cd24dcf9864ff250341d153 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:17.588545 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c328eeb-fdce-4cb6-9d56-709d168e1b98 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:17.589055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c328eeb-fdce-4cb6-9d56-709d168e1b98 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:17.589055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c328eeb-fdce-4cb6-9d56-709d168e1b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:17.607432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-7c328eeb-fdce-4cb6-9d56-709d168e1b98 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:17.608660 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-7c328eeb-fdce-4cb6-9d56-709d168e1b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:17.677694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1295/2583] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:17.689446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7e9b1-78a8-4a65-92f6-779fdb301182 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:17.691177 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b4b7e9b1-78a8-4a65-92f6-779fdb301182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:17.754868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7e9b1-78a8-4a65-92f6-779fdb301182 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:17.759405 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e6695-64c7-4fed-afd2-3eb3ce4739ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:17.760054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e6695-64c7-4fed-afd2-3eb3ce4739ae service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:17.760335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e6695-64c7-4fed-afd2-3eb3ce4739ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:17.760609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e6695-64c7-4fed-afd2-3eb3ce4739ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:17.770064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e6695-64c7-4fed-afd2-3eb3ce4739ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:17.771534 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e6695-64c7-4fed-afd2-3eb3ce4739ae service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:17.781093 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1289/2584] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:51:18.273814 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c328eeb-fdce-4cb6-9d56-709d168e1b98 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:18.275400 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-7c328eeb-fdce-4cb6-9d56-709d168e1b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:18.288056 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c328eeb-fdce-4cb6-9d56-709d168e1b98 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:18.293782 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace3dc9-a0aa-4b35-9d7e-21cca734d492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:18.294805 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace3dc9-a0aa-4b35-9d7e-21cca734d492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:18.295090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace3dc9-a0aa-4b35-9d7e-21cca734d492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:18.295359 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace3dc9-a0aa-4b35-9d7e-21cca734d492 admin admin] PATH_INFO: `/v3/projects/0b28b45a23e540b597b4d83d3b99ddd4` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:18.296591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace3dc9-a0aa-4b35-9d7e-21cca734d492 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b28b45a23e540b597b4d83d3b99ddd4)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:18.298563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace3dc9-a0aa-4b35-9d7e-21cca734d492 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:18.350646 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-2ace3dc9-a0aa-4b35-9d7e-21cca734d492 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b28b45a23e540b597b4d83d3b99ddd4', 'request_id': 'req-2ace3dc9-a0aa-4b35-9d7e-21cca734d492'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:18.355967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-2ace3dc9-a0aa-4b35-9d7e-21cca734d492 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b28b45a23e540b597b4d83d3b99ddd4', 'request_id': 'req-2ace3dc9-a0aa-4b35-9d7e-21cca734d492'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:18.364601 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-2ace3dc9-a0aa-4b35-9d7e-21cca734d492 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b28b45a23e540b597b4d83d3b99ddd4', 'request_id': 'req-2ace3dc9-a0aa-4b35-9d7e-21cca734d492'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:18.377643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-2ace3dc9-a0aa-4b35-9d7e-21cca734d492 admin admin] The token cache is being invalidate because project 0b28b45a23e540b597b4d83d3b99ddd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:51:18.377932 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-2ace3dc9-a0aa-4b35-9d7e-21cca734d492 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:18.378643 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-2ace3dc9-a0aa-4b35-9d7e-21cca734d492 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:18.380301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1296/2585] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:51:18 2022] DELETE /identity/v3/projects/0b28b45a23e540b597b4d83d3b99ddd4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:18.385369 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a198ccf-e0cc-404d-93fa-62ad94c548b5 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:18.385660 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a198ccf-e0cc-404d-93fa-62ad94c548b5 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:18.385938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a198ccf-e0cc-404d-93fa-62ad94c548b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:18.405273 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-0a198ccf-e0cc-404d-93fa-62ad94c548b5 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:18.406470 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0a198ccf-e0cc-404d-93fa-62ad94c548b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:18.468341 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1290/2586] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:18.476186 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c1884-eec6-4be6-9189-7cd92ff611d8 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:18.476474 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c1884-eec6-4be6-9189-7cd92ff611d8 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:18.476748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c1884-eec6-4be6-9189-7cd92ff611d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:18.496274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-898c1884-eec6-4be6-9189-7cd92ff611d8 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:18.497473 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-898c1884-eec6-4be6-9189-7cd92ff611d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:18.505827 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1297/2587] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:51:18.510282 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a198ccf-e0cc-404d-93fa-62ad94c548b5 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:18.520172 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a198ccf-e0cc-404d-93fa-62ad94c548b5 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:18.525187 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9aafd-23f7-415f-b199-ca6bf081645e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:18.525837 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9aafd-23f7-415f-b199-ca6bf081645e admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:18.526118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9aafd-23f7-415f-b199-ca6bf081645e admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:18.526391 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9aafd-23f7-415f-b199-ca6bf081645e admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:18.527258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9aafd-23f7-415f-b199-ca6bf081645e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:51:18.527565 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9aafd-23f7-415f-b199-ca6bf081645e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:18.529182 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9aafd-23f7-415f-b199-ca6bf081645e admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:18.541091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1291/2588] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:51:18.545484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c1884-eec6-4be6-9189-7cd92ff611d8 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:18.555000 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c1884-eec6-4be6-9189-7cd92ff611d8 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:18.560296 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc71c3-c059-4f93-a0dd-56e5b3e4c9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:18.560945 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc71c3-c059-4f93-a0dd-56e5b3e4c9f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:18.561228 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc71c3-c059-4f93-a0dd-56e5b3e4c9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:18.561504 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc71c3-c059-4f93-a0dd-56e5b3e4c9f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:18.562200 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc71c3-c059-4f93-a0dd-56e5b3e4c9f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:18.564201 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc71c3-c059-4f93-a0dd-56e5b3e4c9f7 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:18.578206 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:18.595232 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1298/2589] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:51:18 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:18.599005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9aafd-23f7-415f-b199-ca6bf081645e admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:18.609323 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9aafd-23f7-415f-b199-ca6bf081645e admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:18.617327 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:18.668715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b6448-d9e8-4950-b13c-fd18d20472c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:18.669402 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b6448-d9e8-4950-b13c-fd18d20472c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:18.669683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b6448-d9e8-4950-b13c-fd18d20472c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:18.669961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b6448-d9e8-4950-b13c-fd18d20472c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:18.670652 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b6448-d9e8-4950-b13c-fd18d20472c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:18.672686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b6448-d9e8-4950-b13c-fd18d20472c0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:18.695004 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1292/2590] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:51:18 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:18.699289 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc71c3-c059-4f93-a0dd-56e5b3e4c9f7 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:18.709465 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc71c3-c059-4f93-a0dd-56e5b3e4c9f7 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:18.714561 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f567216-1ccc-40ed-bc3c-5a8de2af9362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:18.715252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f567216-1ccc-40ed-bc3c-5a8de2af9362 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:18.715534 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f567216-1ccc-40ed-bc3c-5a8de2af9362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:18.715808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f567216-1ccc-40ed-bc3c-5a8de2af9362 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:18.716537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f567216-1ccc-40ed-bc3c-5a8de2af9362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:18.718142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f567216-1ccc-40ed-bc3c-5a8de2af9362 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:18.728699 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1299/2591] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:18.732552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b6448-d9e8-4950-b13c-fd18d20472c0 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:18.743439 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b6448-d9e8-4950-b13c-fd18d20472c0 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:18.749202 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a48f8c-49e9-4617-be9c-41afd365bc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:18.749874 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a48f8c-49e9-4617-be9c-41afd365bc06 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:18.750160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a48f8c-49e9-4617-be9c-41afd365bc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:18.750434 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a48f8c-49e9-4617-be9c-41afd365bc06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:18.751199 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a48f8c-49e9-4617-be9c-41afd365bc06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:18.752819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a48f8c-49e9-4617-be9c-41afd365bc06 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:18.763081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1293/2592] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:18.767175 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f567216-1ccc-40ed-bc3c-5a8de2af9362 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:18.777136 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f567216-1ccc-40ed-bc3c-5a8de2af9362 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:18.782561 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3788d-b066-449b-9862-a53704185523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:18.783471 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3788d-b066-449b-9862-a53704185523 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:18.783755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3788d-b066-449b-9862-a53704185523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:18.784033 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d3788d-b066-449b-9862-a53704185523 admin admin] PATH_INFO: `/v3/projects/c8470ff1fe1f45508fb895a2eadf0353/users/89221021169e4a3198f126b737a0a4ed/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:18.798620 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3788d-b066-449b-9862-a53704185523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8470ff1fe1f45508fb895a2eadf0353, user_id=89221021169e4a3198f126b737a0a4ed, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:18.801336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d3788d-b066-449b-9862-a53704185523 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:18.818189 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1300/2593] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:51:18 2022] PUT /identity/v3/projects/c8470ff1fe1f45508fb895a2eadf0353/users/89221021169e4a3198f126b737a0a4ed/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:18.830266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19a71a-035f-4221-94e7-866791035683 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:18.830553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19a71a-035f-4221-94e7-866791035683 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:18.830831 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19a71a-035f-4221-94e7-866791035683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:18.858837 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-bd19a71a-035f-4221-94e7-866791035683 None None] MFA Rules not processed for user `89221021169e4a3198f126b737a0a4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:18.860092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-bd19a71a-035f-4221-94e7-866791035683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:18.938172 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1294/2594] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:18.942867 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3788d-b066-449b-9862-a53704185523 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:19.015938 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d3788d-b066-449b-9862-a53704185523 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:19.021123 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1121f3-6e51-4bab-87cb-e804b9c362bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:19.021762 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1121f3-6e51-4bab-87cb-e804b9c362bd admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:19.022043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1121f3-6e51-4bab-87cb-e804b9c362bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:19.022317 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1121f3-6e51-4bab-87cb-e804b9c362bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:19.023002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1121f3-6e51-4bab-87cb-e804b9c362bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:19.024955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1121f3-6e51-4bab-87cb-e804b9c362bd admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:19.054466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1301/2595] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:51:18 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:19.058657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19a71a-035f-4221-94e7-866791035683 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:19.069771 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19a71a-035f-4221-94e7-866791035683 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:19.126424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe9926-c043-4b2d-a234-0b261a70f91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:19.127147 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe9926-c043-4b2d-a234-0b261a70f91f admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:19.127440 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe9926-c043-4b2d-a234-0b261a70f91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:19.127715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe9926-c043-4b2d-a234-0b261a70f91f admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:19.128407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe9926-c043-4b2d-a234-0b261a70f91f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:19.130349 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe9926-c043-4b2d-a234-0b261a70f91f admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:19.151194 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1295/2596] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:51:19 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:19.155533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1121f3-6e51-4bab-87cb-e804b9c362bd admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:19.166472 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1121f3-6e51-4bab-87cb-e804b9c362bd admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:19.171239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f486b0-bcb9-4a60-84eb-2efaef7641ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:19.171909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f486b0-bcb9-4a60-84eb-2efaef7641ca admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:19.172193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f486b0-bcb9-4a60-84eb-2efaef7641ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:19.172471 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f486b0-bcb9-4a60-84eb-2efaef7641ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:19.173204 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f486b0-bcb9-4a60-84eb-2efaef7641ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:19.174833 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f486b0-bcb9-4a60-84eb-2efaef7641ca admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:19.186720 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1302/2597] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:19.190956 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe9926-c043-4b2d-a234-0b261a70f91f admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:19.201449 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe9926-c043-4b2d-a234-0b261a70f91f admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:19.206410 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a1897-f2ec-40d8-82a0-42281afac758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:19.207493 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a1897-f2ec-40d8-82a0-42281afac758 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:19.207782 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a1897-f2ec-40d8-82a0-42281afac758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:19.208133 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a1897-f2ec-40d8-82a0-42281afac758 admin admin] PATH_INFO: `/v3/projects/d902d9a6eb2a42ff840b242a94c93534/users/c3fa2699709f4ae9b189ad20d1c7979b/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:19.212598 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:19.226188 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a1897-f2ec-40d8-82a0-42281afac758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d902d9a6eb2a42ff840b242a94c93534, user_id=c3fa2699709f4ae9b189ad20d1c7979b, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:19.229109 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a1897-f2ec-40d8-82a0-42281afac758 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:19.243124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1296/2598] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:51:19 2022] PUT /identity/v3/projects/d902d9a6eb2a42ff840b242a94c93534/users/c3fa2699709f4ae9b189ad20d1c7979b/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:19.247261 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f486b0-bcb9-4a60-84eb-2efaef7641ca admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:19.257401 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f486b0-bcb9-4a60-84eb-2efaef7641ca admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:19.315867 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa96f4a8-82af-43d9-a211-3d5b69db65b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:19.316462 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96f4a8-82af-43d9-a211-3d5b69db65b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:19.316745 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96f4a8-82af-43d9-a211-3d5b69db65b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:19.317021 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa96f4a8-82af-43d9-a211-3d5b69db65b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:19.317734 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96f4a8-82af-43d9-a211-3d5b69db65b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:19.319427 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa96f4a8-82af-43d9-a211-3d5b69db65b5 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:19.332071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1303/2599] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:19.336839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a1897-f2ec-40d8-82a0-42281afac758 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:19.349602 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a1897-f2ec-40d8-82a0-42281afac758 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:19.357439 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f9290-e819-4355-94b7-33d2f282a4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:19.358339 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f9290-e819-4355-94b7-33d2f282a4fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:19.358602 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f9290-e819-4355-94b7-33d2f282a4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:19.358873 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f9290-e819-4355-94b7-33d2f282a4fe admin admin] PATH_INFO: `/v3/projects/d902d9a6eb2a42ff840b242a94c93534/users/c3fa2699709f4ae9b189ad20d1c7979b/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:19.361263 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f9290-e819-4355-94b7-33d2f282a4fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d902d9a6eb2a42ff840b242a94c93534, user_id=c3fa2699709f4ae9b189ad20d1c7979b, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:19.363993 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f9290-e819-4355-94b7-33d2f282a4fe admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:19.376081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1297/2600] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:51:19 2022] PUT /identity/v3/projects/d902d9a6eb2a42ff840b242a94c93534/users/c3fa2699709f4ae9b189ad20d1c7979b/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:19.391914 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda6694-1d54-41af-ba78-8c5baa98b124 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:19.392210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda6694-1d54-41af-ba78-8c5baa98b124 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:19.392473 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda6694-1d54-41af-ba78-8c5baa98b124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:19.420408 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-6dda6694-1d54-41af-ba78-8c5baa98b124 None None] MFA Rules not processed for user `c3fa2699709f4ae9b189ad20d1c7979b`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:19.421595 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6dda6694-1d54-41af-ba78-8c5baa98b124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:19.439221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:19.491034 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1304/2601] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:19.503188 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f9290-e819-4355-94b7-33d2f282a4fe admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:19.505096 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-1d7f9290-e819-4355-94b7-33d2f282a4fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:19.564155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f9290-e819-4355-94b7-33d2f282a4fe admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:19.568499 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b39092-43b6-410b-9578-0b45bf03f159 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:19.569095 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b39092-43b6-410b-9578-0b45bf03f159 service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:19.569389 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b39092-43b6-410b-9578-0b45bf03f159 service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:19.569669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b39092-43b6-410b-9578-0b45bf03f159 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:19.578871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b39092-43b6-410b-9578-0b45bf03f159 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:19.580362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b39092-43b6-410b-9578-0b45bf03f159 service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:19.590108 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1298/2602] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:51:19.983401 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda6694-1d54-41af-ba78-8c5baa98b124 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:20.049050 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda6694-1d54-41af-ba78-8c5baa98b124 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:20.053909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ed0ab-ec0b-449d-9548-c21142217f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:20.054790 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ed0ab-ec0b-449d-9548-c21142217f8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:20.055099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ed0ab-ec0b-449d-9548-c21142217f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:20.055363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ed0ab-ec0b-449d-9548-c21142217f8f admin admin] PATH_INFO: `/v3/users/89221021169e4a3198f126b737a0a4ed` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:20.056567 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ed0ab-ec0b-449d-9548-c21142217f8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=89221021169e4a3198f126b737a0a4ed)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:20.058609 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ed0ab-ec0b-449d-9548-c21142217f8f admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:20.113631 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a64ed0ab-ec0b-449d-9548-c21142217f8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89221021169e4a3198f126b737a0a4ed'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:20.125333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a64ed0ab-ec0b-449d-9548-c21142217f8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89221021169e4a3198f126b737a0a4ed'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:20.132205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a64ed0ab-ec0b-449d-9548-c21142217f8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89221021169e4a3198f126b737a0a4ed'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:20.132871 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a64ed0ab-ec0b-449d-9548-c21142217f8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89221021169e4a3198f126b737a0a4ed'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:20.133470 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-a64ed0ab-ec0b-449d-9548-c21142217f8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89221021169e4a3198f126b737a0a4ed'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:20.152264 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1305/2603] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:51:19 2022] DELETE /identity/v3/users/89221021169e4a3198f126b737a0a4ed => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:20.157495 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3b6be-40f9-4b11-bcac-ee1333584e98 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:20.157783 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3b6be-40f9-4b11-bcac-ee1333584e98 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:20.158059 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3b6be-40f9-4b11-bcac-ee1333584e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:20.176442 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-f3e3b6be-40f9-4b11-bcac-ee1333584e98 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:20.177673 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f3e3b6be-40f9-4b11-bcac-ee1333584e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:20.243814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1299/2604] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:20.253854 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ed0ab-ec0b-449d-9548-c21142217f8f admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:20.255573 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a64ed0ab-ec0b-449d-9548-c21142217f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:20.292593 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:20.318760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ed0ab-ec0b-449d-9548-c21142217f8f admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:20.322993 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6569e-cf46-4b3d-8377-5674ce3a56c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:20.323667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6569e-cf46-4b3d-8377-5674ce3a56c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:20.323976 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6569e-cf46-4b3d-8377-5674ce3a56c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:20.324218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6569e-cf46-4b3d-8377-5674ce3a56c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:20.334251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6569e-cf46-4b3d-8377-5674ce3a56c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:20.335728 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6569e-cf46-4b3d-8377-5674ce3a56c9 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:20.345177 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1306/2605] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:51:20.618887 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3b6be-40f9-4b11-bcac-ee1333584e98 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:20.620371 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f3e3b6be-40f9-4b11-bcac-ee1333584e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:20.632619 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3b6be-40f9-4b11-bcac-ee1333584e98 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:20.637893 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c307c13-067b-4c06-a622-e7ed0953da80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:20.638774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c307c13-067b-4c06-a622-e7ed0953da80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:20.639083 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c307c13-067b-4c06-a622-e7ed0953da80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:20.639363 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c307c13-067b-4c06-a622-e7ed0953da80 admin admin] PATH_INFO: `/v3/projects/c8470ff1fe1f45508fb895a2eadf0353` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:20.640570 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c307c13-067b-4c06-a622-e7ed0953da80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8470ff1fe1f45508fb895a2eadf0353)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:20.642538 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c307c13-067b-4c06-a622-e7ed0953da80 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:20.692971 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9c307c13-067b-4c06-a622-e7ed0953da80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8470ff1fe1f45508fb895a2eadf0353', 'request_id': 'req-9c307c13-067b-4c06-a622-e7ed0953da80'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:20.699228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9c307c13-067b-4c06-a622-e7ed0953da80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8470ff1fe1f45508fb895a2eadf0353', 'request_id': 'req-9c307c13-067b-4c06-a622-e7ed0953da80'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:20.704979 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9c307c13-067b-4c06-a622-e7ed0953da80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8470ff1fe1f45508fb895a2eadf0353', 'request_id': 'req-9c307c13-067b-4c06-a622-e7ed0953da80'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:20.715041 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:20.720778 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9c307c13-067b-4c06-a622-e7ed0953da80 admin admin] The token cache is being invalidate because project c8470ff1fe1f45508fb895a2eadf0353 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:51:20.721070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9c307c13-067b-4c06-a622-e7ed0953da80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:20.721820 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-9c307c13-067b-4c06-a622-e7ed0953da80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:20.723516 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1300/2606] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:51:20 2022] DELETE /identity/v3/projects/c8470ff1fe1f45508fb895a2eadf0353 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:20.727756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6569e-cf46-4b3d-8377-5674ce3a56c9 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:20.737902 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6569e-cf46-4b3d-8377-5674ce3a56c9 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:20.799529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5af91b-cd6b-4459-b0b9-b2d8ffd399d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:20.800419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5af91b-cd6b-4459-b0b9-b2d8ffd399d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:20.800732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5af91b-cd6b-4459-b0b9-b2d8ffd399d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:20.800998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5af91b-cd6b-4459-b0b9-b2d8ffd399d7 admin admin] PATH_INFO: `/v3/users/c3fa2699709f4ae9b189ad20d1c7979b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:20.802356 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5af91b-cd6b-4459-b0b9-b2d8ffd399d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3fa2699709f4ae9b189ad20d1c7979b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:20.804366 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5af91b-cd6b-4459-b0b9-b2d8ffd399d7 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:20.853188 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-9f5af91b-cd6b-4459-b0b9-b2d8ffd399d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3fa2699709f4ae9b189ad20d1c7979b'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:20.872874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-9f5af91b-cd6b-4459-b0b9-b2d8ffd399d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3fa2699709f4ae9b189ad20d1c7979b'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:20.880232 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-9f5af91b-cd6b-4459-b0b9-b2d8ffd399d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3fa2699709f4ae9b189ad20d1c7979b'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:20.880998 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-9f5af91b-cd6b-4459-b0b9-b2d8ffd399d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3fa2699709f4ae9b189ad20d1c7979b'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:20.881625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-9f5af91b-cd6b-4459-b0b9-b2d8ffd399d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3fa2699709f4ae9b189ad20d1c7979b'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:20.897139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1307/2607] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:51:20 2022] DELETE /identity/v3/users/c3fa2699709f4ae9b189ad20d1c7979b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:51:21.184451 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c307c13-067b-4c06-a622-e7ed0953da80 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:21.186073 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-9c307c13-067b-4c06-a622-e7ed0953da80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:21.200608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c307c13-067b-4c06-a622-e7ed0953da80 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:21.259576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdc329-5491-4329-9b00-388d54b67dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:21.260355 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdc329-5491-4329-9b00-388d54b67dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:21.260632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdc329-5491-4329-9b00-388d54b67dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:21.260905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdc329-5491-4329-9b00-388d54b67dad admin admin] PATH_INFO: `/v3/projects/d902d9a6eb2a42ff840b242a94c93534` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:21.262095 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdc329-5491-4329-9b00-388d54b67dad admin admin] RBAC: Authorizing `identity:delete_project(project_id=d902d9a6eb2a42ff840b242a94c93534)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:21.264127 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdc329-5491-4329-9b00-388d54b67dad admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:21.310669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-01bdc329-5491-4329-9b00-388d54b67dad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd902d9a6eb2a42ff840b242a94c93534', 'request_id': 'req-01bdc329-5491-4329-9b00-388d54b67dad'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:21.316620 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-01bdc329-5491-4329-9b00-388d54b67dad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd902d9a6eb2a42ff840b242a94c93534', 'request_id': 'req-01bdc329-5491-4329-9b00-388d54b67dad'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:21.322165 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-01bdc329-5491-4329-9b00-388d54b67dad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd902d9a6eb2a42ff840b242a94c93534', 'request_id': 'req-01bdc329-5491-4329-9b00-388d54b67dad'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:21.335731 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-01bdc329-5491-4329-9b00-388d54b67dad admin admin] The token cache is being invalidate because project d902d9a6eb2a42ff840b242a94c93534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:51:21.336016 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-01bdc329-5491-4329-9b00-388d54b67dad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:21.336684 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-01bdc329-5491-4329-9b00-388d54b67dad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:21.338204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1301/2608] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:51:21 2022] DELETE /identity/v3/projects/d902d9a6eb2a42ff840b242a94c93534 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:21.344478 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bad83-f5d8-4624-8a87-28c381e03c34 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:21.344962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bad83-f5d8-4624-8a87-28c381e03c34 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:21.345091 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bad83-f5d8-4624-8a87-28c381e03c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:21.363073 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-e33bad83-f5d8-4624-8a87-28c381e03c34 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:21.364272 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-e33bad83-f5d8-4624-8a87-28c381e03c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:21.414484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:21.427786 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1308/2609] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:21.435838 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0cb85-10ac-4992-85c0-adf03e23ffe8 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:21.436130 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0cb85-10ac-4992-85c0-adf03e23ffe8 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:21.436393 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0cb85-10ac-4992-85c0-adf03e23ffe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:21.455344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-8ae0cb85-10ac-4992-85c0-adf03e23ffe8 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:21.456571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8ae0cb85-10ac-4992-85c0-adf03e23ffe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:21.464918 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1302/2610] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:21.469227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bad83-f5d8-4624-8a87-28c381e03c34 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:21.478944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bad83-f5d8-4624-8a87-28c381e03c34 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:21.483964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758f299-cb0f-4a56-b73f-6e211cae1753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:21.484641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758f299-cb0f-4a56-b73f-6e211cae1753 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:21.484927 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758f299-cb0f-4a56-b73f-6e211cae1753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:21.485183 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758f299-cb0f-4a56-b73f-6e211cae1753 admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:21.486160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758f299-cb0f-4a56-b73f-6e211cae1753 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:51:21.486496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758f299-cb0f-4a56-b73f-6e211cae1753 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:21.488167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758f299-cb0f-4a56-b73f-6e211cae1753 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:21.499863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1309/2611] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:51:21.504197 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0cb85-10ac-4992-85c0-adf03e23ffe8 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:21.513773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0cb85-10ac-4992-85c0-adf03e23ffe8 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:21.519688 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29c0c7-0c9b-4a47-8770-4ae1acb41870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:21.520379 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29c0c7-0c9b-4a47-8770-4ae1acb41870 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:21.520661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29c0c7-0c9b-4a47-8770-4ae1acb41870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:21.520937 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29c0c7-0c9b-4a47-8770-4ae1acb41870 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:21.521639 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29c0c7-0c9b-4a47-8770-4ae1acb41870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:21.523638 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29c0c7-0c9b-4a47-8770-4ae1acb41870 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:21.551880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1303/2612] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:51:21 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:51:21.556332 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758f299-cb0f-4a56-b73f-6e211cae1753 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:21.567264 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758f299-cb0f-4a56-b73f-6e211cae1753 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:21.624879 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14e8b9-42a0-4bd3-8a0e-cdb564061859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:21.625552 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14e8b9-42a0-4bd3-8a0e-cdb564061859 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:21.625832 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14e8b9-42a0-4bd3-8a0e-cdb564061859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:21.626103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14e8b9-42a0-4bd3-8a0e-cdb564061859 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:21.626786 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14e8b9-42a0-4bd3-8a0e-cdb564061859 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:21.628764 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14e8b9-42a0-4bd3-8a0e-cdb564061859 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:21.650700 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1310/2613] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:51:21 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:21.654870 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29c0c7-0c9b-4a47-8770-4ae1acb41870 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:21.666088 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29c0c7-0c9b-4a47-8770-4ae1acb41870 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:21.671646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b4e1d-027a-4ca5-aa7e-53d4333fa036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:21.672315 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b4e1d-027a-4ca5-aa7e-53d4333fa036 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:21.672598 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b4e1d-027a-4ca5-aa7e-53d4333fa036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:21.672862 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b4e1d-027a-4ca5-aa7e-53d4333fa036 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:21.673590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b4e1d-027a-4ca5-aa7e-53d4333fa036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:21.675233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b4e1d-027a-4ca5-aa7e-53d4333fa036 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:21.686032 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1304/2614] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:21.690274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14e8b9-42a0-4bd3-8a0e-cdb564061859 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:21.700888 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14e8b9-42a0-4bd3-8a0e-cdb564061859 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:21.706474 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c34a67-c4f4-4bc1-8288-eefc1de1b97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:21.707146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c34a67-c4f4-4bc1-8288-eefc1de1b97b admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:21.707425 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c34a67-c4f4-4bc1-8288-eefc1de1b97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:21.707691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c34a67-c4f4-4bc1-8288-eefc1de1b97b admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:21.708419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c34a67-c4f4-4bc1-8288-eefc1de1b97b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:21.710036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c34a67-c4f4-4bc1-8288-eefc1de1b97b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:21.723272 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1311/2615] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:21.728893 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b4e1d-027a-4ca5-aa7e-53d4333fa036 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:21.739634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b4e1d-027a-4ca5-aa7e-53d4333fa036 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:21.745446 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68d3a7-c09c-40d3-baf9-e6553338fd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:21.746394 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68d3a7-c09c-40d3-baf9-e6553338fd42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:21.746755 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68d3a7-c09c-40d3-baf9-e6553338fd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:21.747044 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68d3a7-c09c-40d3-baf9-e6553338fd42 admin admin] PATH_INFO: `/v3/projects/05c4657f86584836af8aec76d2cc18ab/users/010cc2c85d354f2cb890e96559a11ca9/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:21.763662 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68d3a7-c09c-40d3-baf9-e6553338fd42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05c4657f86584836af8aec76d2cc18ab, user_id=010cc2c85d354f2cb890e96559a11ca9, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:21.766398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68d3a7-c09c-40d3-baf9-e6553338fd42 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:21.781024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1305/2616] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:51:21 2022] PUT /identity/v3/projects/05c4657f86584836af8aec76d2cc18ab/users/010cc2c85d354f2cb890e96559a11ca9/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:21.792971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c7672-a0eb-4cc4-b455-51b80b095077 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:21.793288 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c7672-a0eb-4cc4-b455-51b80b095077 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:21.793564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c7672-a0eb-4cc4-b455-51b80b095077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:21.820054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-051c7672-a0eb-4cc4-b455-51b80b095077 None None] MFA Rules not processed for user `010cc2c85d354f2cb890e96559a11ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:21.821259 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-051c7672-a0eb-4cc4-b455-51b80b095077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:21.885539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1312/2617] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:21.890368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68d3a7-c09c-40d3-baf9-e6553338fd42 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:21.926713 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:21.964633 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68d3a7-c09c-40d3-baf9-e6553338fd42 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:21.969911 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c45e66-b7d8-4ad6-8815-f5e3dcc3d4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:21.970579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c45e66-b7d8-4ad6-8815-f5e3dcc3d4eb admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:21.970864 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c45e66-b7d8-4ad6-8815-f5e3dcc3d4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:21.971172 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c45e66-b7d8-4ad6-8815-f5e3dcc3d4eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:21.971866 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c45e66-b7d8-4ad6-8815-f5e3dcc3d4eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:21.973781 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c45e66-b7d8-4ad6-8815-f5e3dcc3d4eb admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:22.006224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1306/2618] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:51:21 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:22.010630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c7672-a0eb-4cc4-b455-51b80b095077 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:22.022015 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c7672-a0eb-4cc4-b455-51b80b095077 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:22.071759 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:22.084688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0b75c-1b68-4601-9a7b-f16fee0453d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:22.085372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0b75c-1b68-4601-9a7b-f16fee0453d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:22.085659 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0b75c-1b68-4601-9a7b-f16fee0453d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:22.086549 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0b75c-1b68-4601-9a7b-f16fee0453d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:22.086717 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0b75c-1b68-4601-9a7b-f16fee0453d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:22.088610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0b75c-1b68-4601-9a7b-f16fee0453d7 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:22.113340 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1313/2619] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:51:22 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:22.117653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c45e66-b7d8-4ad6-8815-f5e3dcc3d4eb admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:22.128603 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c45e66-b7d8-4ad6-8815-f5e3dcc3d4eb admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:22.134316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e7244-a065-4634-9373-0416c7943e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:22.135142 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e7244-a065-4634-9373-0416c7943e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:22.135404 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e7244-a065-4634-9373-0416c7943e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:22.135671 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e7244-a065-4634-9373-0416c7943e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:22.136398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e7244-a065-4634-9373-0416c7943e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:22.138003 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e7244-a065-4634-9373-0416c7943e3d admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:22.148942 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1307/2620] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:22.153188 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0b75c-1b68-4601-9a7b-f16fee0453d7 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:22.164157 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0b75c-1b68-4601-9a7b-f16fee0453d7 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:22.169084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab9201-e137-4ef7-9c6f-abce6f2337eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:22.169983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab9201-e137-4ef7-9c6f-abce6f2337eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:22.170263 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab9201-e137-4ef7-9c6f-abce6f2337eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:22.170537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab9201-e137-4ef7-9c6f-abce6f2337eb admin admin] PATH_INFO: `/v3/projects/5eb746937983488e9c46303e57c20bc9/users/39abcc9668dd41fdba0176b80d8f95b7/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:22.186703 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab9201-e137-4ef7-9c6f-abce6f2337eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eb746937983488e9c46303e57c20bc9, user_id=39abcc9668dd41fdba0176b80d8f95b7, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:22.189424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab9201-e137-4ef7-9c6f-abce6f2337eb admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:22.205535 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1314/2621] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:51:22 2022] PUT /identity/v3/projects/5eb746937983488e9c46303e57c20bc9/users/39abcc9668dd41fdba0176b80d8f95b7/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:22.209591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e7244-a065-4634-9373-0416c7943e3d admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:22.219927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e7244-a065-4634-9373-0416c7943e3d admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:22.284338 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c31cbb-80d7-43e6-8c2f-080354c448cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:22.284933 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c31cbb-80d7-43e6-8c2f-080354c448cc admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:22.285215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c31cbb-80d7-43e6-8c2f-080354c448cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:22.285492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c31cbb-80d7-43e6-8c2f-080354c448cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:22.286218 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c31cbb-80d7-43e6-8c2f-080354c448cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:22.287873 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c31cbb-80d7-43e6-8c2f-080354c448cc admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:22.297781 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1308/2622] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:22.302154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab9201-e137-4ef7-9c6f-abce6f2337eb admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:22.313543 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab9201-e137-4ef7-9c6f-abce6f2337eb admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:22.319182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc8735-92bf-4f10-8c0e-8ccbe61a912e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:22.320068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc8735-92bf-4f10-8c0e-8ccbe61a912e admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:22.320348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc8735-92bf-4f10-8c0e-8ccbe61a912e admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:22.320642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc8735-92bf-4f10-8c0e-8ccbe61a912e admin admin] PATH_INFO: `/v3/projects/5eb746937983488e9c46303e57c20bc9/users/39abcc9668dd41fdba0176b80d8f95b7/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:22.322619 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc8735-92bf-4f10-8c0e-8ccbe61a912e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eb746937983488e9c46303e57c20bc9, user_id=39abcc9668dd41fdba0176b80d8f95b7, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:22.325304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc8735-92bf-4f10-8c0e-8ccbe61a912e admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:22.338776 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1315/2623] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:51:22 2022] PUT /identity/v3/projects/5eb746937983488e9c46303e57c20bc9/users/39abcc9668dd41fdba0176b80d8f95b7/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:22.349437 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d379f0f-6829-457e-8711-4a7bbccc96f8 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:22.349723 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d379f0f-6829-457e-8711-4a7bbccc96f8 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:22.350001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d379f0f-6829-457e-8711-4a7bbccc96f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:22.377433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-7d379f0f-6829-457e-8711-4a7bbccc96f8 None None] MFA Rules not processed for user `39abcc9668dd41fdba0176b80d8f95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:22.378620 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7d379f0f-6829-457e-8711-4a7bbccc96f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:22.449520 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1309/2624] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:22.461869 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc8735-92bf-4f10-8c0e-8ccbe61a912e admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:22.464139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-0bfc8735-92bf-4f10-8c0e-8ccbe61a912e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:22.524431 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc8735-92bf-4f10-8c0e-8ccbe61a912e admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:22.529827 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3766c73-df5e-40ab-a1ef-f49d62d480c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:22.530581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3766c73-df5e-40ab-a1ef-f49d62d480c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:22.530861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3766c73-df5e-40ab-a1ef-f49d62d480c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:22.531335 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3766c73-df5e-40ab-a1ef-f49d62d480c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:22.541323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3766c73-df5e-40ab-a1ef-f49d62d480c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:22.542738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3766c73-df5e-40ab-a1ef-f49d62d480c0 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:22.552426 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1316/2625] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:51:22.870024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d379f0f-6829-457e-8711-4a7bbccc96f8 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:22.880322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d379f0f-6829-457e-8711-4a7bbccc96f8 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:22.887452 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033418a4-ed63-41d0-9db3-db00eb202f81 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39abcc9668dd41fdba0176b80d8f95b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5eb746937983488e9c46303e57c20bc9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:22.888438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033418a4-ed63-41d0-9db3-db00eb202f81 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:22.888825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033418a4-ed63-41d0-9db3-db00eb202f81 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:22.889195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033418a4-ed63-41d0-9db3-db00eb202f81 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:22.890219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033418a4-ed63-41d0-9db3-db00eb202f81 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:51:22.890653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033418a4-ed63-41d0-9db3-db00eb202f81 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:22.892877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033418a4-ed63-41d0-9db3-db00eb202f81 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:22.906610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1310/2626] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:51:22.911319 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3766c73-df5e-40ab-a1ef-f49d62d480c0 service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:22.920716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3766c73-df5e-40ab-a1ef-f49d62d480c0 service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:22.927174 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc8917-7ca5-46a3-8f3b-a979f02af108 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39abcc9668dd41fdba0176b80d8f95b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5eb746937983488e9c46303e57c20bc9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:22.927737 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc8917-7ca5-46a3-8f3b-a979f02af108 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:22.928020 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc8917-7ca5-46a3-8f3b-a979f02af108 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:22.928290 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc8917-7ca5-46a3-8f3b-a979f02af108 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:22.928982 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc8917-7ca5-46a3-8f3b-a979f02af108 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:22.931142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc8917-7ca5-46a3-8f3b-a979f02af108 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:22.965863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1317/2627] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:51:22 2022] POST /identity/v3/projects => generated 353 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:22.970286 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033418a4-ed63-41d0-9db3-db00eb202f81 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:22.986363 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033418a4-ed63-41d0-9db3-db00eb202f81 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:22.992085 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:23.049122 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d719f1-91f5-4894-9ce3-b793cefccd74 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39abcc9668dd41fdba0176b80d8f95b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5eb746937983488e9c46303e57c20bc9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:23.049797 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d719f1-91f5-4894-9ce3-b793cefccd74 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:23.050083 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d719f1-91f5-4894-9ce3-b793cefccd74 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:23.050362 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d719f1-91f5-4894-9ce3-b793cefccd74 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:23.051134 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d719f1-91f5-4894-9ce3-b793cefccd74 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:51:23.051466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d719f1-91f5-4894-9ce3-b793cefccd74 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:23.053151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d719f1-91f5-4894-9ce3-b793cefccd74 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:23.065119 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1311/2628] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:51:23.102107 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc8917-7ca5-46a3-8f3b-a979f02af108 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:23.112794 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc8917-7ca5-46a3-8f3b-a979f02af108 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:23.118092 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0fcaf-7818-48ae-9b9e-68551a784003 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39abcc9668dd41fdba0176b80d8f95b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5eb746937983488e9c46303e57c20bc9', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:23.118957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0fcaf-7818-48ae-9b9e-68551a784003 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:23.119283 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0fcaf-7818-48ae-9b9e-68551a784003 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:23.119558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0fcaf-7818-48ae-9b9e-68551a784003 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] PATH_INFO: `/v3/projects/2404bd1dfb2e434da2eb55b39acc9cb8` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:23.120840 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0fcaf-7818-48ae-9b9e-68551a784003 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2404bd1dfb2e434da2eb55b39acc9cb8)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:23.122890 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0fcaf-7818-48ae-9b9e-68551a784003 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:23.181951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6fc0fcaf-7818-48ae-9b9e-68551a784003 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2404bd1dfb2e434da2eb55b39acc9cb8', 'request_id': 'req-6fc0fcaf-7818-48ae-9b9e-68551a784003'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:23.189723 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6fc0fcaf-7818-48ae-9b9e-68551a784003 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2404bd1dfb2e434da2eb55b39acc9cb8', 'request_id': 'req-6fc0fcaf-7818-48ae-9b9e-68551a784003'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:23.199475 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6fc0fcaf-7818-48ae-9b9e-68551a784003 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2404bd1dfb2e434da2eb55b39acc9cb8', 'request_id': 'req-6fc0fcaf-7818-48ae-9b9e-68551a784003'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:23.210931 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6fc0fcaf-7818-48ae-9b9e-68551a784003 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] The token cache is being invalidate because project 2404bd1dfb2e434da2eb55b39acc9cb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:51:23.211253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6fc0fcaf-7818-48ae-9b9e-68551a784003 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:23.211968 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6fc0fcaf-7818-48ae-9b9e-68551a784003 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:23.213621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1318/2629] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:51:23 2022] DELETE /identity/v3/projects/2404bd1dfb2e434da2eb55b39acc9cb8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:23.412724 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d719f1-91f5-4894-9ce3-b793cefccd74 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:23.415406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-62d719f1-91f5-4894-9ce3-b793cefccd74 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:23.484949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d719f1-91f5-4894-9ce3-b793cefccd74 tempest-VolumeQuotaClassesTest-1232681357 tempest-VolumeQuotaClassesTest-1232681357-project-admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:23.552590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835637fa-3c74-4bd0-b297-a7edb47015cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:23.553682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835637fa-3c74-4bd0-b297-a7edb47015cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:23.553969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835637fa-3c74-4bd0-b297-a7edb47015cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:23.554337 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835637fa-3c74-4bd0-b297-a7edb47015cb admin admin] PATH_INFO: `/v3/users/010cc2c85d354f2cb890e96559a11ca9` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:23.555751 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835637fa-3c74-4bd0-b297-a7edb47015cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=010cc2c85d354f2cb890e96559a11ca9)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:23.557816 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835637fa-3c74-4bd0-b297-a7edb47015cb admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:23.607489 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-835637fa-3c74-4bd0-b297-a7edb47015cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '010cc2c85d354f2cb890e96559a11ca9'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:23.609932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:23.623526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-835637fa-3c74-4bd0-b297-a7edb47015cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '010cc2c85d354f2cb890e96559a11ca9'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:23.630182 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-835637fa-3c74-4bd0-b297-a7edb47015cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '010cc2c85d354f2cb890e96559a11ca9'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:23.630878 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-835637fa-3c74-4bd0-b297-a7edb47015cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '010cc2c85d354f2cb890e96559a11ca9'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:23.631513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-835637fa-3c74-4bd0-b297-a7edb47015cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '010cc2c85d354f2cb890e96559a11ca9'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:23.645354 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1312/2630] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:51:23 2022] DELETE /identity/v3/users/010cc2c85d354f2cb890e96559a11ca9 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:23.650167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de5cd1-2513-4840-b94c-29d876903b32 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:23.650664 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de5cd1-2513-4840-b94c-29d876903b32 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:23.650799 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de5cd1-2513-4840-b94c-29d876903b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:23.664268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:23.671493 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-45de5cd1-2513-4840-b94c-29d876903b32 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:23.673346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-45de5cd1-2513-4840-b94c-29d876903b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:23.732525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1319/2631] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:23.742633 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835637fa-3c74-4bd0-b297-a7edb47015cb admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:23.744869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-835637fa-3c74-4bd0-b297-a7edb47015cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:23.805354 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835637fa-3c74-4bd0-b297-a7edb47015cb admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:23.809552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f6a41-1aef-400a-ae15-1a5e1980fac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:23.810206 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f6a41-1aef-400a-ae15-1a5e1980fac5 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:23.810476 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f6a41-1aef-400a-ae15-1a5e1980fac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:23.810757 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f6a41-1aef-400a-ae15-1a5e1980fac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:23.820259 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f6a41-1aef-400a-ae15-1a5e1980fac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:23.821709 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f6a41-1aef-400a-ae15-1a5e1980fac5 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:23.831499 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1313/2632] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:51:24.113122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de5cd1-2513-4840-b94c-29d876903b32 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:24.114588 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-45de5cd1-2513-4840-b94c-29d876903b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:24.127512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de5cd1-2513-4840-b94c-29d876903b32 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:24.132255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63364d0f-a106-4a84-a07a-8b7a628d1600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:24.133165 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63364d0f-a106-4a84-a07a-8b7a628d1600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:24.133442 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63364d0f-a106-4a84-a07a-8b7a628d1600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:24.133717 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63364d0f-a106-4a84-a07a-8b7a628d1600 admin admin] PATH_INFO: `/v3/projects/05c4657f86584836af8aec76d2cc18ab` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:24.135076 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63364d0f-a106-4a84-a07a-8b7a628d1600 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05c4657f86584836af8aec76d2cc18ab)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:24.137098 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63364d0f-a106-4a84-a07a-8b7a628d1600 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:24.185304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-63364d0f-a106-4a84-a07a-8b7a628d1600 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05c4657f86584836af8aec76d2cc18ab', 'request_id': 'req-63364d0f-a106-4a84-a07a-8b7a628d1600'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:24.190777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-63364d0f-a106-4a84-a07a-8b7a628d1600 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05c4657f86584836af8aec76d2cc18ab', 'request_id': 'req-63364d0f-a106-4a84-a07a-8b7a628d1600'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:24.199419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-63364d0f-a106-4a84-a07a-8b7a628d1600 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05c4657f86584836af8aec76d2cc18ab', 'request_id': 'req-63364d0f-a106-4a84-a07a-8b7a628d1600'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:24.210668 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-63364d0f-a106-4a84-a07a-8b7a628d1600 admin admin] The token cache is being invalidate because project 05c4657f86584836af8aec76d2cc18ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:51:24.210951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-63364d0f-a106-4a84-a07a-8b7a628d1600 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:24.211651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-63364d0f-a106-4a84-a07a-8b7a628d1600 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:24.213236 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1320/2633] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:51:24 2022] DELETE /identity/v3/projects/05c4657f86584836af8aec76d2cc18ab => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:24.217518 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f6a41-1aef-400a-ae15-1a5e1980fac5 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:24.227871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f6a41-1aef-400a-ae15-1a5e1980fac5 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:24.287129 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca34dd5-ade5-40f9-acbc-845bec82790f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:24.288015 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca34dd5-ade5-40f9-acbc-845bec82790f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:24.288291 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca34dd5-ade5-40f9-acbc-845bec82790f admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:24.288561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca34dd5-ade5-40f9-acbc-845bec82790f admin admin] PATH_INFO: `/v3/users/39abcc9668dd41fdba0176b80d8f95b7` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:24.289813 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca34dd5-ade5-40f9-acbc-845bec82790f admin admin] RBAC: Authorizing `identity:delete_user(user_id=39abcc9668dd41fdba0176b80d8f95b7)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:24.291832 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca34dd5-ade5-40f9-acbc-845bec82790f admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:24.343472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-fca34dd5-ade5-40f9-acbc-845bec82790f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39abcc9668dd41fdba0176b80d8f95b7'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:24.355597 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-fca34dd5-ade5-40f9-acbc-845bec82790f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39abcc9668dd41fdba0176b80d8f95b7'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:24.361292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:24.363785 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-fca34dd5-ade5-40f9-acbc-845bec82790f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39abcc9668dd41fdba0176b80d8f95b7'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:24.364505 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-fca34dd5-ade5-40f9-acbc-845bec82790f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39abcc9668dd41fdba0176b80d8f95b7'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:24.365266 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-fca34dd5-ade5-40f9-acbc-845bec82790f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39abcc9668dd41fdba0176b80d8f95b7'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:24.379380 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1314/2634] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:51:24 2022] DELETE /identity/v3/users/39abcc9668dd41fdba0176b80d8f95b7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:24.652688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63364d0f-a106-4a84-a07a-8b7a628d1600 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:24.654313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-63364d0f-a106-4a84-a07a-8b7a628d1600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:24.669661 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63364d0f-a106-4a84-a07a-8b7a628d1600 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:24.734939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c390e-fee0-43e0-8e6e-8bc87ecaae7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:24.735876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c390e-fee0-43e0-8e6e-8bc87ecaae7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:24.736160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c390e-fee0-43e0-8e6e-8bc87ecaae7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:24.736445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c390e-fee0-43e0-8e6e-8bc87ecaae7e admin admin] PATH_INFO: `/v3/projects/5eb746937983488e9c46303e57c20bc9` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:24.737684 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8c390e-fee0-43e0-8e6e-8bc87ecaae7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eb746937983488e9c46303e57c20bc9)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:24.739662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8c390e-fee0-43e0-8e6e-8bc87ecaae7e admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:24.776083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:24.789558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5f8c390e-fee0-43e0-8e6e-8bc87ecaae7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eb746937983488e9c46303e57c20bc9', 'request_id': 'req-5f8c390e-fee0-43e0-8e6e-8bc87ecaae7e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:24.795862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5f8c390e-fee0-43e0-8e6e-8bc87ecaae7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eb746937983488e9c46303e57c20bc9', 'request_id': 'req-5f8c390e-fee0-43e0-8e6e-8bc87ecaae7e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:24.805050 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5f8c390e-fee0-43e0-8e6e-8bc87ecaae7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eb746937983488e9c46303e57c20bc9', 'request_id': 'req-5f8c390e-fee0-43e0-8e6e-8bc87ecaae7e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:24.817525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5f8c390e-fee0-43e0-8e6e-8bc87ecaae7e admin admin] The token cache is being invalidate because project 5eb746937983488e9c46303e57c20bc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:51:24.817844 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5f8c390e-fee0-43e0-8e6e-8bc87ecaae7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:24.818554 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-5f8c390e-fee0-43e0-8e6e-8bc87ecaae7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:24.820241 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1321/2635] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:51:24 2022] DELETE /identity/v3/projects/5eb746937983488e9c46303e57c20bc9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:24.825582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67364d1c-c6d5-4f6f-8f69-2bd2ef2ad73c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:24.825873 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67364d1c-c6d5-4f6f-8f69-2bd2ef2ad73c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:24.826151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67364d1c-c6d5-4f6f-8f69-2bd2ef2ad73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:24.844344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-67364d1c-c6d5-4f6f-8f69-2bd2ef2ad73c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:24.845548 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-67364d1c-c6d5-4f6f-8f69-2bd2ef2ad73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:24.907478 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1315/2636] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:24.915221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce0a0f-f59e-405e-8a8f-fd709eaafc75 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:24.915511 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce0a0f-f59e-405e-8a8f-fd709eaafc75 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:24.915766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce0a0f-f59e-405e-8a8f-fd709eaafc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:24.934975 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-dfce0a0f-f59e-405e-8a8f-fd709eaafc75 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:24.936223 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-dfce0a0f-f59e-405e-8a8f-fd709eaafc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:24.944590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1322/2637] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:24.948810 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67364d1c-c6d5-4f6f-8f69-2bd2ef2ad73c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:24.958759 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67364d1c-c6d5-4f6f-8f69-2bd2ef2ad73c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:24.963649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5a1ba-f6f5-4d0d-ab51-eac76bcb71ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:24.964303 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5a1ba-f6f5-4d0d-ab51-eac76bcb71ff admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:24.964582 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5a1ba-f6f5-4d0d-ab51-eac76bcb71ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:24.964858 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5a1ba-f6f5-4d0d-ab51-eac76bcb71ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:24.965612 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5a1ba-f6f5-4d0d-ab51-eac76bcb71ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:51:24.965914 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5a1ba-f6f5-4d0d-ab51-eac76bcb71ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:24.967548 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5a1ba-f6f5-4d0d-ab51-eac76bcb71ff admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:24.979495 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1316/2638] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:51:24.984613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce0a0f-f59e-405e-8a8f-fd709eaafc75 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:24.999084 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce0a0f-f59e-405e-8a8f-fd709eaafc75 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:25.008429 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656e76a-671b-41b3-9765-2e7139ff3fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:25.009358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656e76a-671b-41b3-9765-2e7139ff3fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:25.009733 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656e76a-671b-41b3-9765-2e7139ff3fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:25.010071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656e76a-671b-41b3-9765-2e7139ff3fc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:25.011070 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656e76a-671b-41b3-9765-2e7139ff3fc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:25.013595 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656e76a-671b-41b3-9765-2e7139ff3fc6 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:25.043872 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1323/2639] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:51:24 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:25.048236 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5a1ba-f6f5-4d0d-ab51-eac76bcb71ff admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:25.058649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5a1ba-f6f5-4d0d-ab51-eac76bcb71ff admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:25.118871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50985f93-6c66-477a-9576-ba4e203a3547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:25.119571 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50985f93-6c66-477a-9576-ba4e203a3547 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:25.119845 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50985f93-6c66-477a-9576-ba4e203a3547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:25.120121 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50985f93-6c66-477a-9576-ba4e203a3547 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:25.120811 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50985f93-6c66-477a-9576-ba4e203a3547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:25.122815 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50985f93-6c66-477a-9576-ba4e203a3547 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:25.145349 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1317/2640] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:51:25 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:25.149603 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656e76a-671b-41b3-9765-2e7139ff3fc6 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:25.160546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656e76a-671b-41b3-9765-2e7139ff3fc6 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:25.165513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708a7bd-694e-413a-ab70-19aeade3b682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:25.166171 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708a7bd-694e-413a-ab70-19aeade3b682 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:25.166455 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708a7bd-694e-413a-ab70-19aeade3b682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:25.166731 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708a7bd-694e-413a-ab70-19aeade3b682 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:25.167523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708a7bd-694e-413a-ab70-19aeade3b682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:25.169212 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708a7bd-694e-413a-ab70-19aeade3b682 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:25.179481 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1324/2641] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:25.183696 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50985f93-6c66-477a-9576-ba4e203a3547 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:25.194214 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50985f93-6c66-477a-9576-ba4e203a3547 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:25.199670 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3d319-c25f-412a-a825-b7afbdf07f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:25.200628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3d319-c25f-412a-a825-b7afbdf07f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:25.200628 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3d319-c25f-412a-a825-b7afbdf07f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:25.200830 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3d319-c25f-412a-a825-b7afbdf07f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:25.201562 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3d319-c25f-412a-a825-b7afbdf07f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:25.203205 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3d319-c25f-412a-a825-b7afbdf07f65 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:25.213604 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1318/2642] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:25.218079 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708a7bd-694e-413a-ab70-19aeade3b682 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:25.228417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708a7bd-694e-413a-ab70-19aeade3b682 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:25.233748 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac017365-d2fa-4ade-af9a-053ba008fc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:25.234638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac017365-d2fa-4ade-af9a-053ba008fc98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:25.234922 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac017365-d2fa-4ade-af9a-053ba008fc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:25.235248 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac017365-d2fa-4ade-af9a-053ba008fc98 admin admin] PATH_INFO: `/v3/projects/90541db9877041ed88db94a93591d4ba/users/a31170a3026f4666826ce5853272c33f/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:25.251944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac017365-d2fa-4ade-af9a-053ba008fc98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90541db9877041ed88db94a93591d4ba, user_id=a31170a3026f4666826ce5853272c33f, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:25.254790 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac017365-d2fa-4ade-af9a-053ba008fc98 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:25.269662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1325/2643] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:51:25 2022] PUT /identity/v3/projects/90541db9877041ed88db94a93591d4ba/users/a31170a3026f4666826ce5853272c33f/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:25.281666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3ccd8-fc6c-4d04-8bcd-db0275e2bc13 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:25.282552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3ccd8-fc6c-4d04-8bcd-db0275e2bc13 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:25.282668 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3ccd8-fc6c-4d04-8bcd-db0275e2bc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:25.313425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-bba3ccd8-fc6c-4d04-8bcd-db0275e2bc13 None None] MFA Rules not processed for user `a31170a3026f4666826ce5853272c33f`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:25.314637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-bba3ccd8-fc6c-4d04-8bcd-db0275e2bc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:25.361224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:25.383912 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1319/2644] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:51:25.388634 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac017365-d2fa-4ade-af9a-053ba008fc98 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:25.454564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac017365-d2fa-4ade-af9a-053ba008fc98 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:25.459690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacc905-1a39-4e28-be92-2e05b015a9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:25.460355 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacc905-1a39-4e28-be92-2e05b015a9aa admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:25.460640 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacc905-1a39-4e28-be92-2e05b015a9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:25.460916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacc905-1a39-4e28-be92-2e05b015a9aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:25.461617 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacc905-1a39-4e28-be92-2e05b015a9aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:25.463599 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacc905-1a39-4e28-be92-2e05b015a9aa admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:25.491911 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1326/2645] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:51:25 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:25.496064 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3ccd8-fc6c-4d04-8bcd-db0275e2bc13 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:25.507479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3ccd8-fc6c-4d04-8bcd-db0275e2bc13 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:25.565751 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71b095-fe5e-4bff-916a-4ceac52ea691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:25.566589 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71b095-fe5e-4bff-916a-4ceac52ea691 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:25.566942 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71b095-fe5e-4bff-916a-4ceac52ea691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:25.567255 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71b095-fe5e-4bff-916a-4ceac52ea691 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:25.568091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71b095-fe5e-4bff-916a-4ceac52ea691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:25.570082 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71b095-fe5e-4bff-916a-4ceac52ea691 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:25.592434 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1320/2646] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:51:25 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:25.596770 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacc905-1a39-4e28-be92-2e05b015a9aa admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:25.607877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacc905-1a39-4e28-be92-2e05b015a9aa admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:25.613450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818e41f-50a2-4025-bf62-6c39659778c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:25.614106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818e41f-50a2-4025-bf62-6c39659778c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:25.614386 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818e41f-50a2-4025-bf62-6c39659778c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:25.614664 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818e41f-50a2-4025-bf62-6c39659778c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:25.615417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818e41f-50a2-4025-bf62-6c39659778c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:25.617011 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818e41f-50a2-4025-bf62-6c39659778c4 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:25.628381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1327/2647] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:25.632650 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71b095-fe5e-4bff-916a-4ceac52ea691 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:25.643390 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71b095-fe5e-4bff-916a-4ceac52ea691 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:25.648630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52addc-1110-4701-9215-17e9c09e3815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:25.649539 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52addc-1110-4701-9215-17e9c09e3815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:25.649827 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52addc-1110-4701-9215-17e9c09e3815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:25.650109 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52addc-1110-4701-9215-17e9c09e3815 admin admin] PATH_INFO: `/v3/projects/8cf669344625468492d1d068ba4a762c/users/1f090bd5989d47a8a009836479c6284c/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:25.668335 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52addc-1110-4701-9215-17e9c09e3815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cf669344625468492d1d068ba4a762c, user_id=1f090bd5989d47a8a009836479c6284c, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:25.671129 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52addc-1110-4701-9215-17e9c09e3815 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:25.685916 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1321/2648] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:51:25 2022] PUT /identity/v3/projects/8cf669344625468492d1d068ba4a762c/users/1f090bd5989d47a8a009836479c6284c/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:25.690198 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818e41f-50a2-4025-bf62-6c39659778c4 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:25.701849 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818e41f-50a2-4025-bf62-6c39659778c4 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:25.732325 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:25.762586 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e9512-2afa-45e0-beb6-f4e4f5de6bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:25.763326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e9512-2afa-45e0-beb6-f4e4f5de6bbf admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:25.763616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e9512-2afa-45e0-beb6-f4e4f5de6bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:25.763895 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e9512-2afa-45e0-beb6-f4e4f5de6bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:25.764641 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e9512-2afa-45e0-beb6-f4e4f5de6bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:25.766270 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e9512-2afa-45e0-beb6-f4e4f5de6bbf admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:25.777002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1328/2649] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:25.781668 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52addc-1110-4701-9215-17e9c09e3815 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:25.793417 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52addc-1110-4701-9215-17e9c09e3815 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:25.798296 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390dbc8-a268-4a07-ae7e-8882d05b935c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:25.799311 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390dbc8-a268-4a07-ae7e-8882d05b935c admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:25.799663 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390dbc8-a268-4a07-ae7e-8882d05b935c admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:25.800017 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390dbc8-a268-4a07-ae7e-8882d05b935c admin admin] PATH_INFO: `/v3/projects/8cf669344625468492d1d068ba4a762c/users/1f090bd5989d47a8a009836479c6284c/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:25.802224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390dbc8-a268-4a07-ae7e-8882d05b935c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cf669344625468492d1d068ba4a762c, user_id=1f090bd5989d47a8a009836479c6284c, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:25.804935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390dbc8-a268-4a07-ae7e-8882d05b935c admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:25.816669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1322/2650] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:51:25 2022] PUT /identity/v3/projects/8cf669344625468492d1d068ba4a762c/users/1f090bd5989d47a8a009836479c6284c/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:25.826971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f8b50-6ce4-4d11-9dd1-933bc9b06b7b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:25.827301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f8b50-6ce4-4d11-9dd1-933bc9b06b7b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:25.827586 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f8b50-6ce4-4d11-9dd1-933bc9b06b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:25.855498 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-d37f8b50-6ce4-4d11-9dd1-933bc9b06b7b None None] MFA Rules not processed for user `1f090bd5989d47a8a009836479c6284c`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:25.856749 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d37f8b50-6ce4-4d11-9dd1-933bc9b06b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:25.925375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1329/2651] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:25.936549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390dbc8-a268-4a07-ae7e-8882d05b935c admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:25.938445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d390dbc8-a268-4a07-ae7e-8882d05b935c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:25.996747 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390dbc8-a268-4a07-ae7e-8882d05b935c admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:26.000576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc1c5b-394c-43de-90c3-c78b1b7f6a6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:26.001228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc1c5b-394c-43de-90c3-c78b1b7f6a6e service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:26.001519 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc1c5b-394c-43de-90c3-c78b1b7f6a6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:26.001774 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc1c5b-394c-43de-90c3-c78b1b7f6a6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:26.011393 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc1c5b-394c-43de-90c3-c78b1b7f6a6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:26.012881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc1c5b-394c-43de-90c3-c78b1b7f6a6e service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:26.022670 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1323/2652] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:51:26.155249 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f8b50-6ce4-4d11-9dd1-933bc9b06b7b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:26.313539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f8b50-6ce4-4d11-9dd1-933bc9b06b7b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:26.324761 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cd2f1-03ac-4fca-82af-1227081efd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:26.325771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cd2f1-03ac-4fca-82af-1227081efd16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:26.326053 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cd2f1-03ac-4fca-82af-1227081efd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:26.326326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cd2f1-03ac-4fca-82af-1227081efd16 admin admin] PATH_INFO: `/v3/users/a31170a3026f4666826ce5853272c33f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:26.329455 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66cd2f1-03ac-4fca-82af-1227081efd16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a31170a3026f4666826ce5853272c33f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:26.331529 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66cd2f1-03ac-4fca-82af-1227081efd16 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:26.422254 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b66cd2f1-03ac-4fca-82af-1227081efd16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a31170a3026f4666826ce5853272c33f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:26.428970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:26.438564 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b66cd2f1-03ac-4fca-82af-1227081efd16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a31170a3026f4666826ce5853272c33f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:26.449501 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b66cd2f1-03ac-4fca-82af-1227081efd16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a31170a3026f4666826ce5853272c33f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:26.450386 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b66cd2f1-03ac-4fca-82af-1227081efd16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a31170a3026f4666826ce5853272c33f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:26.451680 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b66cd2f1-03ac-4fca-82af-1227081efd16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a31170a3026f4666826ce5853272c33f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:26.488069 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1330/2653] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:51:26 2022] DELETE /identity/v3/users/a31170a3026f4666826ce5853272c33f => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:26.492678 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75477450-f19a-4444-871e-58a2a164e2e7 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:26.493202 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75477450-f19a-4444-871e-58a2a164e2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:26.493298 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75477450-f19a-4444-871e-58a2a164e2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:26.515841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-75477450-f19a-4444-871e-58a2a164e2e7 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:26.517303 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-75477450-f19a-4444-871e-58a2a164e2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:26.582164 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:26.619841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1324/2654] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:51:26.659803 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cd2f1-03ac-4fca-82af-1227081efd16 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:26.662233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b66cd2f1-03ac-4fca-82af-1227081efd16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:26.727496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cd2f1-03ac-4fca-82af-1227081efd16 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:26.731233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace557a-2df7-41e4-b376-04466c6e0629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:26.731979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace557a-2df7-41e4-b376-04466c6e0629 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:26.732339 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace557a-2df7-41e4-b376-04466c6e0629 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:26.732739 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace557a-2df7-41e4-b376-04466c6e0629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:26.742509 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace557a-2df7-41e4-b376-04466c6e0629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:26.744075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace557a-2df7-41e4-b376-04466c6e0629 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:26.755524 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1331/2655] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:51:27.160114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75477450-f19a-4444-871e-58a2a164e2e7 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:27.161669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-75477450-f19a-4444-871e-58a2a164e2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:27.176750 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75477450-f19a-4444-871e-58a2a164e2e7 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:27.181794 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:27.182836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:27.183159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:27.183442 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b admin admin] PATH_INFO: `/v3/projects/90541db9877041ed88db94a93591d4ba` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:27.185447 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=90541db9877041ed88db94a93591d4ba)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:27.187503 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:27.242702 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90541db9877041ed88db94a93591d4ba', 'request_id': 'req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:27.249006 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90541db9877041ed88db94a93591d4ba', 'request_id': 'req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:27.255209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90541db9877041ed88db94a93591d4ba', 'request_id': 'req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:27.268070 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b admin admin] The token cache is being invalidate because project 90541db9877041ed88db94a93591d4ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:51:27.268361 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:27.269314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:27.271475 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1325/2656] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:51:27 2022] DELETE /identity/v3/projects/90541db9877041ed88db94a93591d4ba => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:27.277402 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace557a-2df7-41e4-b376-04466c6e0629 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:27.287125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace557a-2df7-41e4-b376-04466c6e0629 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:27.348921 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13b10e-c6dd-49a2-81db-8d065dd425d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:27.349801 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13b10e-c6dd-49a2-81db-8d065dd425d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:27.350077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13b10e-c6dd-49a2-81db-8d065dd425d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:27.350347 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13b10e-c6dd-49a2-81db-8d065dd425d0 admin admin] PATH_INFO: `/v3/users/1f090bd5989d47a8a009836479c6284c` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:27.351585 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13b10e-c6dd-49a2-81db-8d065dd425d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f090bd5989d47a8a009836479c6284c)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:27.353532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13b10e-c6dd-49a2-81db-8d065dd425d0 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:27.399574 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-aa13b10e-c6dd-49a2-81db-8d065dd425d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f090bd5989d47a8a009836479c6284c'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:27.412909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-aa13b10e-c6dd-49a2-81db-8d065dd425d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f090bd5989d47a8a009836479c6284c'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:27.419903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-aa13b10e-c6dd-49a2-81db-8d065dd425d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f090bd5989d47a8a009836479c6284c'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:27.420619 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-aa13b10e-c6dd-49a2-81db-8d065dd425d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f090bd5989d47a8a009836479c6284c'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:27.421231 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-aa13b10e-c6dd-49a2-81db-8d065dd425d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f090bd5989d47a8a009836479c6284c'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:27.425024 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:27.439869 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1332/2657] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:51:27 2022] DELETE /identity/v3/users/1f090bd5989d47a8a009836479c6284c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:51:27.675826 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:27.677093 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:27.772990 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13b10e-c6dd-49a2-81db-8d065dd425d0 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:27.774527 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-aa13b10e-c6dd-49a2-81db-8d065dd425d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:27.797140 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d28a09-e6cb-49cd-9b14-a66b05f55c6b admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:27.870332 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:27.874210 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13b10e-c6dd-49a2-81db-8d065dd425d0 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:27.879355 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748da9a9-4e5c-41fb-9a7b-588bd4099788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:27.880369 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748da9a9-4e5c-41fb-9a7b-588bd4099788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:27.880753 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748da9a9-4e5c-41fb-9a7b-588bd4099788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:27.881090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748da9a9-4e5c-41fb-9a7b-588bd4099788 admin admin] PATH_INFO: `/v3/projects/8cf669344625468492d1d068ba4a762c` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:27.882373 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748da9a9-4e5c-41fb-9a7b-588bd4099788 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cf669344625468492d1d068ba4a762c)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:27.883865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069a354-4994-41e6-80b5-a2218380a73a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:27.884452 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748da9a9-4e5c-41fb-9a7b-588bd4099788 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:27.885043 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069a354-4994-41e6-80b5-a2218380a73a service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:27.885681 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069a354-4994-41e6-80b5-a2218380a73a service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:27.886049 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069a354-4994-41e6-80b5-a2218380a73a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:27.888048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7069a354-4994-41e6-80b5-a2218380a73a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:27.902351 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7069a354-4994-41e6-80b5-a2218380a73a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:27.904197 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7069a354-4994-41e6-80b5-a2218380a73a service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:27.916475 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1326/2658] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:51:27.934192 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-748da9a9-4e5c-41fb-9a7b-588bd4099788 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cf669344625468492d1d068ba4a762c', 'request_id': 'req-748da9a9-4e5c-41fb-9a7b-588bd4099788'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:27.939791 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-748da9a9-4e5c-41fb-9a7b-588bd4099788 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cf669344625468492d1d068ba4a762c', 'request_id': 'req-748da9a9-4e5c-41fb-9a7b-588bd4099788'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:27.949562 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-748da9a9-4e5c-41fb-9a7b-588bd4099788 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cf669344625468492d1d068ba4a762c', 'request_id': 'req-748da9a9-4e5c-41fb-9a7b-588bd4099788'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:27.966543 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-748da9a9-4e5c-41fb-9a7b-588bd4099788 admin admin] The token cache is being invalidate because project 8cf669344625468492d1d068ba4a762c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:51:27.966875 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-748da9a9-4e5c-41fb-9a7b-588bd4099788 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:27.967621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-748da9a9-4e5c-41fb-9a7b-588bd4099788 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:27.969357 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1333/2659] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:51:27 2022] DELETE /identity/v3/projects/8cf669344625468492d1d068ba4a762c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:27.974209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6a527-f2ea-4aa6-9465-b558a625f0e0 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:27.974500 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6a527-f2ea-4aa6-9465-b558a625f0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:27.974780 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6a527-f2ea-4aa6-9465-b558a625f0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:27.998472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-b4b6a527-f2ea-4aa6-9465-b558a625f0e0 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:27.999923 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b4b6a527-f2ea-4aa6-9465-b558a625f0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:28.060450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1327/2660] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:28.069727 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940150f-224b-43ba-a809-0a14cc3b76ac None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:28.070245 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940150f-224b-43ba-a809-0a14cc3b76ac None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:28.070597 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1940150f-224b-43ba-a809-0a14cc3b76ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:28.091155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-1940150f-224b-43ba-a809-0a14cc3b76ac None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:28.092396 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1940150f-224b-43ba-a809-0a14cc3b76ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:28.102181 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1334/2661] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:28.106715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6a527-f2ea-4aa6-9465-b558a625f0e0 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:28.116826 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6a527-f2ea-4aa6-9465-b558a625f0e0 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:28.121943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692f0e5-2e17-4631-8b18-065bf5f00e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:28.122611 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692f0e5-2e17-4631-8b18-065bf5f00e1b admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:28.122890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692f0e5-2e17-4631-8b18-065bf5f00e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:28.123196 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692f0e5-2e17-4631-8b18-065bf5f00e1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:28.123991 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692f0e5-2e17-4631-8b18-065bf5f00e1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:51:28.124293 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692f0e5-2e17-4631-8b18-065bf5f00e1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:28.125935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4692f0e5-2e17-4631-8b18-065bf5f00e1b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:28.137551 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1328/2662] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:51:28.142082 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940150f-224b-43ba-a809-0a14cc3b76ac None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:28.151143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1940150f-224b-43ba-a809-0a14cc3b76ac None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:28.155876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd4024-5cad-452f-9317-73c0fd5dd9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:28.156515 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd4024-5cad-452f-9317-73c0fd5dd9ac admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:28.157161 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd4024-5cad-452f-9317-73c0fd5dd9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:28.157432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd4024-5cad-452f-9317-73c0fd5dd9ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:28.158133 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd4024-5cad-452f-9317-73c0fd5dd9ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:28.160077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd4024-5cad-452f-9317-73c0fd5dd9ac admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:28.190113 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1335/2663] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:51:28 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:51:28.191415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692f0e5-2e17-4631-8b18-065bf5f00e1b admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:28.202441 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692f0e5-2e17-4631-8b18-065bf5f00e1b admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:28.261210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9b3c0-8727-4dbc-8772-d470693d44bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:28.261907 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9b3c0-8727-4dbc-8772-d470693d44bc admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:28.262270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9b3c0-8727-4dbc-8772-d470693d44bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:28.262669 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9b3c0-8727-4dbc-8772-d470693d44bc admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:28.263429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9b3c0-8727-4dbc-8772-d470693d44bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:28.265407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9b3c0-8727-4dbc-8772-d470693d44bc admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:28.287474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1329/2664] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:51:28 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:28.291790 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd4024-5cad-452f-9317-73c0fd5dd9ac admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:28.303269 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd4024-5cad-452f-9317-73c0fd5dd9ac admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:28.307293 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ae470-18ef-452b-b0cb-a2eb1a8ee09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:28.307986 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ae470-18ef-452b-b0cb-a2eb1a8ee09c admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:28.308316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ae470-18ef-452b-b0cb-a2eb1a8ee09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:28.308736 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ae470-18ef-452b-b0cb-a2eb1a8ee09c admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:28.309559 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ae470-18ef-452b-b0cb-a2eb1a8ee09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:28.311532 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ae470-18ef-452b-b0cb-a2eb1a8ee09c admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:28.324106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1336/2665] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:28.328807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9b3c0-8727-4dbc-8772-d470693d44bc admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:28.341544 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9b3c0-8727-4dbc-8772-d470693d44bc admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:28.347517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a45e7-2f43-4c65-86ff-f6bb2dc5cac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:28.348211 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a45e7-2f43-4c65-86ff-f6bb2dc5cac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:28.348499 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a45e7-2f43-4c65-86ff-f6bb2dc5cac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:28.348778 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a45e7-2f43-4c65-86ff-f6bb2dc5cac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:28.349663 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a45e7-2f43-4c65-86ff-f6bb2dc5cac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:28.351321 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a45e7-2f43-4c65-86ff-f6bb2dc5cac5 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:28.366633 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1330/2666] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:28.367558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ae470-18ef-452b-b0cb-a2eb1a8ee09c admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:28.385015 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ae470-18ef-452b-b0cb-a2eb1a8ee09c admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:28.390448 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96299ab-1b16-449e-9100-01508145b694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:28.391403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96299ab-1b16-449e-9100-01508145b694 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:28.391685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96299ab-1b16-449e-9100-01508145b694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:28.392460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96299ab-1b16-449e-9100-01508145b694 admin admin] PATH_INFO: `/v3/projects/4a6ca476be674ec4a96993f135f66a80/users/ef4ed725d22a4826bd8c7ef0b8c4df67/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:28.412268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96299ab-1b16-449e-9100-01508145b694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a6ca476be674ec4a96993f135f66a80, user_id=ef4ed725d22a4826bd8c7ef0b8c4df67, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:28.416311 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96299ab-1b16-449e-9100-01508145b694 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:28.442059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1337/2667] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:51:28 2022] PUT /identity/v3/projects/4a6ca476be674ec4a96993f135f66a80/users/ef4ed725d22a4826bd8c7ef0b8c4df67/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:28.444963 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf14c7-6886-4745-a061-18a3b7a11267 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:28.445349 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf14c7-6886-4745-a061-18a3b7a11267 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:28.445681 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf14c7-6886-4745-a061-18a3b7a11267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:28.464935 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-ebcf14c7-6886-4745-a061-18a3b7a11267 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:28.466232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ebcf14c7-6886-4745-a061-18a3b7a11267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:28.520249 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:28.536031 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1331/2668] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:28.546605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96299ab-1b16-449e-9100-01508145b694 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:28.553506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a96299ab-1b16-449e-9100-01508145b694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:28.584810 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:28.631044 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96299ab-1b16-449e-9100-01508145b694 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:28.636722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301de409-fb44-4cdf-bd75-fc74d46bf11e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:28.637753 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301de409-fb44-4cdf-bd75-fc74d46bf11e service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:28.638040 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301de409-fb44-4cdf-bd75-fc74d46bf11e service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:28.638441 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301de409-fb44-4cdf-bd75-fc74d46bf11e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:28.650018 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301de409-fb44-4cdf-bd75-fc74d46bf11e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:28.652418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301de409-fb44-4cdf-bd75-fc74d46bf11e service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:28.667376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1338/2669] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:51:29.294852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7014b-5975-40dc-9fcb-6b1f1e075978 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:29.295395 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7014b-5975-40dc-9fcb-6b1f1e075978 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:29.295517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7014b-5975-40dc-9fcb-6b1f1e075978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:29.321220 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-96e7014b-5975-40dc-9fcb-6b1f1e075978 None None] MFA Rules not processed for user `ef4ed725d22a4826bd8c7ef0b8c4df67`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:29.322431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-96e7014b-5975-40dc-9fcb-6b1f1e075978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:29.390487 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1332/2670] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:29.401391 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301de409-fb44-4cdf-bd75-fc74d46bf11e service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:29.402911 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-301de409-fb44-4cdf-bd75-fc74d46bf11e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:29.461296 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301de409-fb44-4cdf-bd75-fc74d46bf11e service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:29.465385 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f85abf-6390-4b52-86a2-a47130095d1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:29.466191 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f85abf-6390-4b52-86a2-a47130095d1c service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:29.466479 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f85abf-6390-4b52-86a2-a47130095d1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:29.466760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f85abf-6390-4b52-86a2-a47130095d1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:29.477382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f85abf-6390-4b52-86a2-a47130095d1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:29.479657 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f85abf-6390-4b52-86a2-a47130095d1c service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:29.489573 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1339/2671] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:51:30.851137 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f8161-c3b7-4823-bb75-8edf21095f90 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:30.851781 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f8161-c3b7-4823-bb75-8edf21095f90 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:30.851781 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f8161-c3b7-4823-bb75-8edf21095f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:30.869393 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-ce3f8161-c3b7-4823-bb75-8edf21095f90 None None] MFA Rules not processed for user `ef4ed725d22a4826bd8c7ef0b8c4df67`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:30.870597 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ce3f8161-c3b7-4823-bb75-8edf21095f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:30.877458 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1333/2672] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:31.126205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f85abf-6390-4b52-86a2-a47130095d1c service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:31.127867 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f2f85abf-6390-4b52-86a2-a47130095d1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:31.140326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f85abf-6390-4b52-86a2-a47130095d1c service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:31.144839 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35273c4-88a7-4389-a23f-ecc0f0e25a24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:31.145481 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35273c4-88a7-4389-a23f-ecc0f0e25a24 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:31.145747 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35273c4-88a7-4389-a23f-ecc0f0e25a24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:31.146024 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35273c4-88a7-4389-a23f-ecc0f0e25a24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:31.147836 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f35273c4-88a7-4389-a23f-ecc0f0e25a24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:31.158136 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35273c4-88a7-4389-a23f-ecc0f0e25a24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:31.159644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35273c4-88a7-4389-a23f-ecc0f0e25a24 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:31.167838 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1340/2673] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:51:33.069992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da5065-6004-41a2-b1a3-afd12460bbfb None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:33.070476 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da5065-6004-41a2-b1a3-afd12460bbfb None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:33.070611 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da5065-6004-41a2-b1a3-afd12460bbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:33.088816 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-33da5065-6004-41a2-b1a3-afd12460bbfb None None] MFA Rules not processed for user `0cbf797171144c139c72c516cea59bb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:33.090029 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-33da5065-6004-41a2-b1a3-afd12460bbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:33.151615 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1334/2674] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:34.916771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedaff3-9d1d-4f01-8eaf-be97effea44b None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:34.917203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedaff3-9d1d-4f01-8eaf-be97effea44b None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:34.917328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedaff3-9d1d-4f01-8eaf-be97effea44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:34.936626 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-6dedaff3-9d1d-4f01-8eaf-be97effea44b None None] MFA Rules not processed for user `4d04998b636649d98103b1cd5f3a0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:34.938192 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6dedaff3-9d1d-4f01-8eaf-be97effea44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:34.999769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1341/2675] 10.222.0.44 () {58 vars in 967 bytes} [Thu Jul 21 05:51:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:35.058764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da5065-6004-41a2-b1a3-afd12460bbfb None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:35.061071 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-33da5065-6004-41a2-b1a3-afd12460bbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:35.133882 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da5065-6004-41a2-b1a3-afd12460bbfb None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:35.138873 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4401f32-d47a-43fd-ba41-fe7cb2f96277 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61b61188b5b94a51ba1ae1a88b78b89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:35.139566 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4401f32-d47a-43fd-ba41-fe7cb2f96277 service swift] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:35.139850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4401f32-d47a-43fd-ba41-fe7cb2f96277 service swift] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:35.140125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4401f32-d47a-43fd-ba41-fe7cb2f96277 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:35.148787 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4401f32-d47a-43fd-ba41-fe7cb2f96277 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:35.150487 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4401f32-d47a-43fd-ba41-fe7cb2f96277 service swift] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:35.159701 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1335/2676] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jul 21 05:51:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 05:51:39.020052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedaff3-9d1d-4f01-8eaf-be97effea44b None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:39.024149 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6dedaff3-9d1d-4f01-8eaf-be97effea44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:39.124746 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:39.141243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedaff3-9d1d-4f01-8eaf-be97effea44b None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:39.148360 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4152996-290e-4ef6-9b56-bac560d4afa3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:39.149029 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4152996-290e-4ef6-9b56-bac560d4afa3 service placement] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:39.149314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4152996-290e-4ef6-9b56-bac560d4afa3 service placement] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:39.149591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4152996-290e-4ef6-9b56-bac560d4afa3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:39.151453 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c4152996-290e-4ef6-9b56-bac560d4afa3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:39.177997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4152996-290e-4ef6-9b56-bac560d4afa3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:39.179720 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4152996-290e-4ef6-9b56-bac560d4afa3 service placement] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:39.190438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1342/2677] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:51:42.921990 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4401f32-d47a-43fd-ba41-fe7cb2f96277 service swift] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:42.923572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b4401f32-d47a-43fd-ba41-fe7cb2f96277 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:42.945704 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:43.010456 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4401f32-d47a-43fd-ba41-fe7cb2f96277 service swift] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:43.015678 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65222124-631b-418b-a322-14ef07d92970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:43.016768 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65222124-631b-418b-a322-14ef07d92970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:43.017154 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65222124-631b-418b-a322-14ef07d92970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:43.017661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65222124-631b-418b-a322-14ef07d92970 admin admin] PATH_INFO: `/v3/users/22b285ea44c0494d9fa611df20d66a27` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:43.018931 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65222124-631b-418b-a322-14ef07d92970 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22b285ea44c0494d9fa611df20d66a27)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:43.021081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65222124-631b-418b-a322-14ef07d92970 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:43.084959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-65222124-631b-418b-a322-14ef07d92970 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22b285ea44c0494d9fa611df20d66a27'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:43.108347 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-65222124-631b-418b-a322-14ef07d92970 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22b285ea44c0494d9fa611df20d66a27'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:43.116114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-65222124-631b-418b-a322-14ef07d92970 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22b285ea44c0494d9fa611df20d66a27'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:43.116879 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-65222124-631b-418b-a322-14ef07d92970 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22b285ea44c0494d9fa611df20d66a27'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:43.117504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-65222124-631b-418b-a322-14ef07d92970 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22b285ea44c0494d9fa611df20d66a27'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:43.132848 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1336/2678] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:51:42 2022] DELETE /identity/v3/users/22b285ea44c0494d9fa611df20d66a27 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:43.410040 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4152996-290e-4ef6-9b56-bac560d4afa3 service placement] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:43.448843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4152996-290e-4ef6-9b56-bac560d4afa3 service placement] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:43.506979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08358d92-36c2-4daf-ac3b-5d2531ade75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:43.507878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08358d92-36c2-4daf-ac3b-5d2531ade75d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:43.508157 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08358d92-36c2-4daf-ac3b-5d2531ade75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:43.508424 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08358d92-36c2-4daf-ac3b-5d2531ade75d admin admin] PATH_INFO: `/v3/projects/c1c0c1c0f91147b98e49acb05e14d2d2` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:43.509630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08358d92-36c2-4daf-ac3b-5d2531ade75d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1c0c1c0f91147b98e49acb05e14d2d2)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:43.511593 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08358d92-36c2-4daf-ac3b-5d2531ade75d admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:43.558943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-08358d92-36c2-4daf-ac3b-5d2531ade75d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1c0c1c0f91147b98e49acb05e14d2d2', 'request_id': 'req-08358d92-36c2-4daf-ac3b-5d2531ade75d'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:43.565916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-08358d92-36c2-4daf-ac3b-5d2531ade75d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1c0c1c0f91147b98e49acb05e14d2d2', 'request_id': 'req-08358d92-36c2-4daf-ac3b-5d2531ade75d'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:43.575508 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-08358d92-36c2-4daf-ac3b-5d2531ade75d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1c0c1c0f91147b98e49acb05e14d2d2', 'request_id': 'req-08358d92-36c2-4daf-ac3b-5d2531ade75d'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:43.586992 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-08358d92-36c2-4daf-ac3b-5d2531ade75d admin admin] The token cache is being invalidate because project c1c0c1c0f91147b98e49acb05e14d2d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:51:43.587259 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-08358d92-36c2-4daf-ac3b-5d2531ade75d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:43.587974 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-08358d92-36c2-4daf-ac3b-5d2531ade75d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:43.589679 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1343/2679] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:51:43 2022] DELETE /identity/v3/projects/c1c0c1c0f91147b98e49acb05e14d2d2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:43.593908 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65222124-631b-418b-a322-14ef07d92970 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:43.595407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-65222124-631b-418b-a322-14ef07d92970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:43.609202 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65222124-631b-418b-a322-14ef07d92970 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:43.679139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de64a93-1cd6-4b96-9ecd-8f1480a04b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:43.680048 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de64a93-1cd6-4b96-9ecd-8f1480a04b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:43.680336 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de64a93-1cd6-4b96-9ecd-8f1480a04b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:43.680612 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de64a93-1cd6-4b96-9ecd-8f1480a04b08 admin admin] PATH_INFO: `/v3/users/053c267e63c041d3808e53950cc67ab7` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:43.681851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de64a93-1cd6-4b96-9ecd-8f1480a04b08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=053c267e63c041d3808e53950cc67ab7)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:43.683855 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de64a93-1cd6-4b96-9ecd-8f1480a04b08 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:43.719194 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:43.732666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-2de64a93-1cd6-4b96-9ecd-8f1480a04b08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '053c267e63c041d3808e53950cc67ab7'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:43.744356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-2de64a93-1cd6-4b96-9ecd-8f1480a04b08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '053c267e63c041d3808e53950cc67ab7'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:43.751034 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-2de64a93-1cd6-4b96-9ecd-8f1480a04b08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '053c267e63c041d3808e53950cc67ab7'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:43.751700 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-2de64a93-1cd6-4b96-9ecd-8f1480a04b08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '053c267e63c041d3808e53950cc67ab7'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:43.752304 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-2de64a93-1cd6-4b96-9ecd-8f1480a04b08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '053c267e63c041d3808e53950cc67ab7'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:43.769585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1337/2680] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:51:43 2022] DELETE /identity/v3/users/053c267e63c041d3808e53950cc67ab7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:44.088943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08358d92-36c2-4daf-ac3b-5d2531ade75d admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:44.090798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-08358d92-36c2-4daf-ac3b-5d2531ade75d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:44.104183 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08358d92-36c2-4daf-ac3b-5d2531ade75d admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:44.168395 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:44.191220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf4ee9-4ce9-4645-b001-597169384719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:44.192105 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf4ee9-4ce9-4645-b001-597169384719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:44.192386 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf4ee9-4ce9-4645-b001-597169384719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:44.192706 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf4ee9-4ce9-4645-b001-597169384719 admin admin] PATH_INFO: `/v3/projects/108552a2639a4fe2ab275355ca6f9e00` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:44.193910 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf4ee9-4ce9-4645-b001-597169384719 admin admin] RBAC: Authorizing `identity:delete_project(project_id=108552a2639a4fe2ab275355ca6f9e00)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:44.196257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf4ee9-4ce9-4645-b001-597169384719 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:44.253450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-37cf4ee9-4ce9-4645-b001-597169384719 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '108552a2639a4fe2ab275355ca6f9e00', 'request_id': 'req-37cf4ee9-4ce9-4645-b001-597169384719'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:44.260127 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-37cf4ee9-4ce9-4645-b001-597169384719 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '108552a2639a4fe2ab275355ca6f9e00', 'request_id': 'req-37cf4ee9-4ce9-4645-b001-597169384719'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:44.270775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-37cf4ee9-4ce9-4645-b001-597169384719 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '108552a2639a4fe2ab275355ca6f9e00', 'request_id': 'req-37cf4ee9-4ce9-4645-b001-597169384719'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:44.284668 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-37cf4ee9-4ce9-4645-b001-597169384719 admin admin] The token cache is being invalidate because project 108552a2639a4fe2ab275355ca6f9e00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:51:44.285215 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-37cf4ee9-4ce9-4645-b001-597169384719 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:44.285939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-37cf4ee9-4ce9-4645-b001-597169384719 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:51:44.287984 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1344/2681] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:51:44 2022] DELETE /identity/v3/projects/108552a2639a4fe2ab275355ca6f9e00 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:44.292801 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d822d-b6e4-4c68-a5fe-62b425886dd2 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:44.293402 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d822d-b6e4-4c68-a5fe-62b425886dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:44.293402 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d822d-b6e4-4c68-a5fe-62b425886dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:44.357154 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-d39d822d-b6e4-4c68-a5fe-62b425886dd2 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:44.358294 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-d39d822d-b6e4-4c68-a5fe-62b425886dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:44.420857 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1338/2682] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:44.428901 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc9583-91d9-442d-bff6-c61de27e34fc None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:44.429440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc9583-91d9-442d-bff6-c61de27e34fc None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:44.429440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc9583-91d9-442d-bff6-c61de27e34fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:44.447619 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-eadc9583-91d9-442d-bff6-c61de27e34fc None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:44.448805 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-eadc9583-91d9-442d-bff6-c61de27e34fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:44.456672 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1345/2683] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:51:44.460824 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d822d-b6e4-4c68-a5fe-62b425886dd2 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:44.470810 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d822d-b6e4-4c68-a5fe-62b425886dd2 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:44.476114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc74567-f959-4de4-812d-0546e261d326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:44.476767 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc74567-f959-4de4-812d-0546e261d326 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:44.477049 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc74567-f959-4de4-812d-0546e261d326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:44.477323 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc74567-f959-4de4-812d-0546e261d326 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:44.478079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc74567-f959-4de4-812d-0546e261d326 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:51:44.478385 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc74567-f959-4de4-812d-0546e261d326 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:44.480052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc74567-f959-4de4-812d-0546e261d326 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:44.492190 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1339/2684] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:51:44.498295 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc9583-91d9-442d-bff6-c61de27e34fc None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:44.508371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc9583-91d9-442d-bff6-c61de27e34fc None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:44.514417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed36d5-b3a0-4429-9c78-ea7a1850f009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:44.515286 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed36d5-b3a0-4429-9c78-ea7a1850f009 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:44.515572 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed36d5-b3a0-4429-9c78-ea7a1850f009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:44.515840 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed36d5-b3a0-4429-9c78-ea7a1850f009 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:44.516545 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed36d5-b3a0-4429-9c78-ea7a1850f009 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:44.518445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed36d5-b3a0-4429-9c78-ea7a1850f009 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:44.548957 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1346/2685] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:51:44 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:51:44.553131 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc74567-f959-4de4-812d-0546e261d326 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:44.563517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc74567-f959-4de4-812d-0546e261d326 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:44.620024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341fd8d-6773-40cf-98c0-b4772590eb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:44.620594 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341fd8d-6773-40cf-98c0-b4772590eb87 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:44.620896 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341fd8d-6773-40cf-98c0-b4772590eb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:44.621169 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341fd8d-6773-40cf-98c0-b4772590eb87 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:44.621940 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341fd8d-6773-40cf-98c0-b4772590eb87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:44.623917 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341fd8d-6773-40cf-98c0-b4772590eb87 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:44.633722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:44.655050 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1340/2686] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:51:44 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:44.659786 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed36d5-b3a0-4429-9c78-ea7a1850f009 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:44.670584 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed36d5-b3a0-4429-9c78-ea7a1850f009 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:44.675518 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9300955a-b1b2-4f4d-8f84-f4319dfad0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:44.676187 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9300955a-b1b2-4f4d-8f84-f4319dfad0bb admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:44.676475 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9300955a-b1b2-4f4d-8f84-f4319dfad0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:44.676754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9300955a-b1b2-4f4d-8f84-f4319dfad0bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:44.677513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9300955a-b1b2-4f4d-8f84-f4319dfad0bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:44.679163 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9300955a-b1b2-4f4d-8f84-f4319dfad0bb admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:44.690109 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1347/2687] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:44.694300 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341fd8d-6773-40cf-98c0-b4772590eb87 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:44.704764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341fd8d-6773-40cf-98c0-b4772590eb87 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:44.709960 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b519d9-1b2f-4464-97a6-1f27cb6e9a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:44.710626 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b519d9-1b2f-4464-97a6-1f27cb6e9a53 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:44.710907 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b519d9-1b2f-4464-97a6-1f27cb6e9a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:44.711220 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b519d9-1b2f-4464-97a6-1f27cb6e9a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:44.711973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b519d9-1b2f-4464-97a6-1f27cb6e9a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:44.713576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b519d9-1b2f-4464-97a6-1f27cb6e9a53 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:44.723928 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1341/2688] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:44.728101 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9300955a-b1b2-4f4d-8f84-f4319dfad0bb admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:44.738681 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9300955a-b1b2-4f4d-8f84-f4319dfad0bb admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:44.744908 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0c1ad-3630-4cb3-8053-b2c9ee8dc971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:44.745790 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0c1ad-3630-4cb3-8053-b2c9ee8dc971 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:44.746090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0c1ad-3630-4cb3-8053-b2c9ee8dc971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:44.746376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0c1ad-3630-4cb3-8053-b2c9ee8dc971 admin admin] PATH_INFO: `/v3/projects/6c683289fa724e71bb7a9f69154c84a1/users/b894a36d43224aba9fc4b14b151e75b2/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:44.763251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0c1ad-3630-4cb3-8053-b2c9ee8dc971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c683289fa724e71bb7a9f69154c84a1, user_id=b894a36d43224aba9fc4b14b151e75b2, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:44.766000 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0c1ad-3630-4cb3-8053-b2c9ee8dc971 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:44.780975 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1348/2689] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:51:44 2022] PUT /identity/v3/projects/6c683289fa724e71bb7a9f69154c84a1/users/b894a36d43224aba9fc4b14b151e75b2/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:44.786484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383af3fa-d3d9-4424-9fbc-122eabb5cd2f None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:44.786775 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383af3fa-d3d9-4424-9fbc-122eabb5cd2f None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:44.787088 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383af3fa-d3d9-4424-9fbc-122eabb5cd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:44.806037 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-383af3fa-d3d9-4424-9fbc-122eabb5cd2f None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:44.807285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-383af3fa-d3d9-4424-9fbc-122eabb5cd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:44.876423 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1342/2690] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:44.886489 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0c1ad-3630-4cb3-8053-b2c9ee8dc971 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:44.888415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-38a0c1ad-3630-4cb3-8053-b2c9ee8dc971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:44.969783 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0c1ad-3630-4cb3-8053-b2c9ee8dc971 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:44.973610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50639c-4e18-40c2-9a3a-6b6d13dc02ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:44.974292 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50639c-4e18-40c2-9a3a-6b6d13dc02ca service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:44.974581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50639c-4e18-40c2-9a3a-6b6d13dc02ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:44.974865 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50639c-4e18-40c2-9a3a-6b6d13dc02ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:44.984289 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50639c-4e18-40c2-9a3a-6b6d13dc02ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:44.985747 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50639c-4e18-40c2-9a3a-6b6d13dc02ca service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:44.989759 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:44.997284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1349/2691] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:51:49.331557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1f44f-7384-4c1f-8511-3e76a131ac87 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:49.332038 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1f44f-7384-4c1f-8511-3e76a131ac87 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:49.332139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1f44f-7384-4c1f-8511-3e76a131ac87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:49.368386 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-c7e1f44f-7384-4c1f-8511-3e76a131ac87 None None] MFA Rules not processed for user `b894a36d43224aba9fc4b14b151e75b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:49.369622 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-c7e1f44f-7384-4c1f-8511-3e76a131ac87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:49.453672 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1343/2692] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:49.457919 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50639c-4e18-40c2-9a3a-6b6d13dc02ca service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:49.471611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50639c-4e18-40c2-9a3a-6b6d13dc02ca service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:49.478898 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10ec68-80fc-4191-bf2a-0a3c9505423d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:49.480563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10ec68-80fc-4191-bf2a-0a3c9505423d admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:49.480563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10ec68-80fc-4191-bf2a-0a3c9505423d admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:49.480563 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10ec68-80fc-4191-bf2a-0a3c9505423d admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:49.481143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10ec68-80fc-4191-bf2a-0a3c9505423d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:49.483160 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10ec68-80fc-4191-bf2a-0a3c9505423d admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:49.547758 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1350/2693] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:51:49 2022] POST /identity/v3/projects => generated 371 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:49.552611 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1f44f-7384-4c1f-8511-3e76a131ac87 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:49.590742 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1f44f-7384-4c1f-8511-3e76a131ac87 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:49.688809 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e21eda-1b1a-4dbb-9b4e-ff7f034c536b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:49.689513 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e21eda-1b1a-4dbb-9b4e-ff7f034c536b admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:49.689801 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e21eda-1b1a-4dbb-9b4e-ff7f034c536b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:49.690080 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e21eda-1b1a-4dbb-9b4e-ff7f034c536b admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:49.690866 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e21eda-1b1a-4dbb-9b4e-ff7f034c536b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:49.693866 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e21eda-1b1a-4dbb-9b4e-ff7f034c536b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:49.727108 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1344/2694] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:51:49 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:51:49.732376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10ec68-80fc-4191-bf2a-0a3c9505423d admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:49.752026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10ec68-80fc-4191-bf2a-0a3c9505423d admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:49.764066 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f69e9b-f303-48fc-a078-8ea490e867c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:49.764796 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f69e9b-f303-48fc-a078-8ea490e867c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:49.765142 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f69e9b-f303-48fc-a078-8ea490e867c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:49.765425 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f69e9b-f303-48fc-a078-8ea490e867c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:49.766176 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f69e9b-f303-48fc-a078-8ea490e867c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:49.767899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f69e9b-f303-48fc-a078-8ea490e867c9 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:49.792918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1351/2695] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:49.797108 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e21eda-1b1a-4dbb-9b4e-ff7f034c536b admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:49.805925 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:49.811327 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e21eda-1b1a-4dbb-9b4e-ff7f034c536b admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:49.818115 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ac706-cad7-41c0-95b2-1b131071e5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:49.819383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ac706-cad7-41c0-95b2-1b131071e5a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:49.820054 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ac706-cad7-41c0-95b2-1b131071e5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:49.820567 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ac706-cad7-41c0-95b2-1b131071e5a0 admin admin] PATH_INFO: `/v3/projects/a0c485077baa456cb7f8ce777f516fa9/users/a9af99a034584fbc8820ad0f863fcc57/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:49.841567 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ac706-cad7-41c0-95b2-1b131071e5a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0c485077baa456cb7f8ce777f516fa9, user_id=a9af99a034584fbc8820ad0f863fcc57, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:49.845884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ac706-cad7-41c0-95b2-1b131071e5a0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:49.867388 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1345/2696] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:51:49 2022] PUT /identity/v3/projects/a0c485077baa456cb7f8ce777f516fa9/users/a9af99a034584fbc8820ad0f863fcc57/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:51:49.871612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f69e9b-f303-48fc-a078-8ea490e867c9 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:49.884349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f69e9b-f303-48fc-a078-8ea490e867c9 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:49.962946 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdaab89-0b43-4797-9b35-34105d7fee20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:49.964042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdaab89-0b43-4797-9b35-34105d7fee20 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:49.964510 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdaab89-0b43-4797-9b35-34105d7fee20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:49.965058 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdaab89-0b43-4797-9b35-34105d7fee20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:49.966155 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdaab89-0b43-4797-9b35-34105d7fee20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:49.968166 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdaab89-0b43-4797-9b35-34105d7fee20 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:49.993686 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ac706-cad7-41c0-95b2-1b131071e5a0 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:49.998695 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1352/2697] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:51:50.002827 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ac706-cad7-41c0-95b2-1b131071e5a0 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:50.007988 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b3939-7c73-4ec8-b986-5edd26d9b942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:50.009018 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b3939-7c73-4ec8-b986-5edd26d9b942 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:50.009376 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b3939-7c73-4ec8-b986-5edd26d9b942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:50.009723 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b3939-7c73-4ec8-b986-5edd26d9b942 admin admin] PATH_INFO: `/v3/projects/a0c485077baa456cb7f8ce777f516fa9/users/a9af99a034584fbc8820ad0f863fcc57/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:50.011976 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b3939-7c73-4ec8-b986-5edd26d9b942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0c485077baa456cb7f8ce777f516fa9, user_id=a9af99a034584fbc8820ad0f863fcc57, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:50.014871 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b3939-7c73-4ec8-b986-5edd26d9b942 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:50.026288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1346/2698] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:51:49 2022] PUT /identity/v3/projects/a0c485077baa456cb7f8ce777f516fa9/users/a9af99a034584fbc8820ad0f863fcc57/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:51:51.121913 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11c155-bc1a-466d-9b93-2e52b155e197 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:51.122545 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11c155-bc1a-466d-9b93-2e52b155e197 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:51.122545 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11c155-bc1a-466d-9b93-2e52b155e197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:51.144239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-fa11c155-bc1a-466d-9b93-2e52b155e197 None None] MFA Rules not processed for user `ef4ed725d22a4826bd8c7ef0b8c4df67`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:51.145688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-fa11c155-bc1a-466d-9b93-2e52b155e197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:51.219099 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1353/2699] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:53.171777 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b3939-7c73-4ec8-b986-5edd26d9b942 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:53.173610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-ee5b3939-7c73-4ec8-b986-5edd26d9b942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:53.370546 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b3939-7c73-4ec8-b986-5edd26d9b942 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:53.376719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c01306-add3-48be-981e-194c6fbe387f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:53.377536 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c01306-add3-48be-981e-194c6fbe387f service placement] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:53.377920 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c01306-add3-48be-981e-194c6fbe387f service placement] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:53.378299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c01306-add3-48be-981e-194c6fbe387f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:53.384997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b0c01306-add3-48be-981e-194c6fbe387f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:53.421494 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c01306-add3-48be-981e-194c6fbe387f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:53.423166 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c01306-add3-48be-981e-194c6fbe387f service placement] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:53.445372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1347/2700] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:51:57.834732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d642416-e4bc-429d-96c5-6b367b600b3e None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:57.835764 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d642416-e4bc-429d-96c5-6b367b600b3e None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:57.836220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d642416-e4bc-429d-96c5-6b367b600b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:57.870352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-5d642416-e4bc-429d-96c5-6b367b600b3e None None] MFA Rules not processed for user `a9af99a034584fbc8820ad0f863fcc57`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:57.871608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5d642416-e4bc-429d-96c5-6b367b600b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:57.949425 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:57.977414 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1354/2701] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:57 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:51:58.004339 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c01306-add3-48be-981e-194c6fbe387f service placement] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:51:58.005851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b0c01306-add3-48be-981e-194c6fbe387f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:58.121250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c01306-add3-48be-981e-194c6fbe387f service placement] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:51:58.126876 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e310d6-b103-40f9-8a64-afda49a71821 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:51:58.127580 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e310d6-b103-40f9-8a64-afda49a71821 service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:58.127855 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e310d6-b103-40f9-8a64-afda49a71821 service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:58.128333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e310d6-b103-40f9-8a64-afda49a71821 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:58.133652 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:51:58.193739 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e310d6-b103-40f9-8a64-afda49a71821 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:51:58.195643 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e310d6-b103-40f9-8a64-afda49a71821 service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:51:58.205977 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1348/2702] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:51:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:51:59.338073 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb06f5c-f034-4d84-b990-5516273ea2f4 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:51:59.338597 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb06f5c-f034-4d84-b990-5516273ea2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:51:59.338716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb06f5c-f034-4d84-b990-5516273ea2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:51:59.356456 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-9eb06f5c-f034-4d84-b990-5516273ea2f4 None None] MFA Rules not processed for user `b894a36d43224aba9fc4b14b151e75b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:51:59.357625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-9eb06f5c-f034-4d84-b990-5516273ea2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:51:59.364933 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1355/2703] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:51:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:52:08.201176 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=24 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:52:09.351110 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=7 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:52:21.270480 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28270b17-e7bc-446d-bef5-115e42969a61 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:21.270955 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28270b17-e7bc-446d-bef5-115e42969a61 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:21.271113 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28270b17-e7bc-446d-bef5-115e42969a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:21.300150 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-28270b17-e7bc-446d-bef5-115e42969a61 None None] MFA Rules not processed for user `4d04998b636649d98103b1cd5f3a0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:52:21.301341 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-28270b17-e7bc-446d-bef5-115e42969a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:52:21.361969 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1349/2704] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jul 21 05:52:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:52:21.374164 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb06f5c-f034-4d84-b990-5516273ea2f4 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:52:21.375833 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-9eb06f5c-f034-4d84-b990-5516273ea2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:52:21.445890 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb06f5c-f034-4d84-b990-5516273ea2f4 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:52:21.451691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e238568-2e3b-4e72-a8c7-5e9f7a404190 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61b61188b5b94a51ba1ae1a88b78b89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:52:21.452339 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e238568-2e3b-4e72-a8c7-5e9f7a404190 service swift] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:21.452624 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e238568-2e3b-4e72-a8c7-5e9f7a404190 service swift] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:21.452900 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e238568-2e3b-4e72-a8c7-5e9f7a404190 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:21.461260 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e238568-2e3b-4e72-a8c7-5e9f7a404190 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:52:21.462867 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e238568-2e3b-4e72-a8c7-5e9f7a404190 service swift] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:52:21.470835 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1356/2705] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jul 21 05:52:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 05:52:31.354302 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=29 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:52:31.467829 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=32 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:52:37.272947 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0229aeb-4d53-4863-9e95-76378d4e41d0 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:37.273424 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0229aeb-4d53-4863-9e95-76378d4e41d0 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:37.273525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0229aeb-4d53-4863-9e95-76378d4e41d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:37.291954 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-a0229aeb-4d53-4863-9e95-76378d4e41d0 None None] MFA Rules not processed for user `4d04998b636649d98103b1cd5f3a0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:52:37.293154 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a0229aeb-4d53-4863-9e95-76378d4e41d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:52:37.301002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1350/2706] 10.222.0.44 () {58 vars in 967 bytes} [Thu Jul 21 05:52:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:52:37.313403 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e238568-2e3b-4e72-a8c7-5e9f7a404190 service swift] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:52:37.324747 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e238568-2e3b-4e72-a8c7-5e9f7a404190 service swift] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:52:37.331274 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76263a-a0dc-4235-8542-33bfaeef813f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61b61188b5b94a51ba1ae1a88b78b89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:52:37.331921 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76263a-a0dc-4235-8542-33bfaeef813f service swift] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:37.332254 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76263a-a0dc-4235-8542-33bfaeef813f service swift] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:37.332534 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76263a-a0dc-4235-8542-33bfaeef813f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:37.341253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76263a-a0dc-4235-8542-33bfaeef813f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:52:37.342850 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76263a-a0dc-4235-8542-33bfaeef813f service swift] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:52:37.351169 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1357/2707] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jul 21 05:52:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 05:52:39.309670 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0229aeb-4d53-4863-9e95-76378d4e41d0 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:52:39.310485 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a0229aeb-4d53-4863-9e95-76378d4e41d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:52:39.381689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0229aeb-4d53-4863-9e95-76378d4e41d0 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:52:39.386512 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e590df-e172-466d-8472-6ea5bf8f5629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:52:39.387460 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e590df-e172-466d-8472-6ea5bf8f5629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:39.387719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e590df-e172-466d-8472-6ea5bf8f5629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:39.387992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e590df-e172-466d-8472-6ea5bf8f5629 admin admin] PATH_INFO: `/v3/users/ef4ed725d22a4826bd8c7ef0b8c4df67` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:39.389294 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e590df-e172-466d-8472-6ea5bf8f5629 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef4ed725d22a4826bd8c7ef0b8c4df67)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:52:39.391267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e590df-e172-466d-8472-6ea5bf8f5629 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:52:39.443662 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-06e590df-e172-466d-8472-6ea5bf8f5629 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef4ed725d22a4826bd8c7ef0b8c4df67'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:39.456891 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-06e590df-e172-466d-8472-6ea5bf8f5629 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef4ed725d22a4826bd8c7ef0b8c4df67'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:39.464701 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-06e590df-e172-466d-8472-6ea5bf8f5629 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef4ed725d22a4826bd8c7ef0b8c4df67'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:39.465768 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-06e590df-e172-466d-8472-6ea5bf8f5629 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef4ed725d22a4826bd8c7ef0b8c4df67'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:39.466528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-06e590df-e172-466d-8472-6ea5bf8f5629 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef4ed725d22a4826bd8c7ef0b8c4df67'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:39.485460 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1351/2708] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:52:39 2022] DELETE /identity/v3/users/ef4ed725d22a4826bd8c7ef0b8c4df67 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:52:39.675208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76263a-a0dc-4235-8542-33bfaeef813f service swift] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:52:39.688183 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76263a-a0dc-4235-8542-33bfaeef813f service swift] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:52:39.747482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885b3a3-0db1-4e95-ad47-97520a7de897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:52:39.748316 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885b3a3-0db1-4e95-ad47-97520a7de897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:39.748603 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885b3a3-0db1-4e95-ad47-97520a7de897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:39.748888 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885b3a3-0db1-4e95-ad47-97520a7de897 admin admin] PATH_INFO: `/v3/projects/4a6ca476be674ec4a96993f135f66a80` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:39.750083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885b3a3-0db1-4e95-ad47-97520a7de897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a6ca476be674ec4a96993f135f66a80)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:52:39.752043 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885b3a3-0db1-4e95-ad47-97520a7de897 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:52:39.803203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f885b3a3-0db1-4e95-ad47-97520a7de897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a6ca476be674ec4a96993f135f66a80', 'request_id': 'req-f885b3a3-0db1-4e95-ad47-97520a7de897'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:39.809093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f885b3a3-0db1-4e95-ad47-97520a7de897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a6ca476be674ec4a96993f135f66a80', 'request_id': 'req-f885b3a3-0db1-4e95-ad47-97520a7de897'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:39.816990 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f885b3a3-0db1-4e95-ad47-97520a7de897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a6ca476be674ec4a96993f135f66a80', 'request_id': 'req-f885b3a3-0db1-4e95-ad47-97520a7de897'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:39.828747 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f885b3a3-0db1-4e95-ad47-97520a7de897 admin admin] The token cache is being invalidate because project 4a6ca476be674ec4a96993f135f66a80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:52:39.829045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f885b3a3-0db1-4e95-ad47-97520a7de897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:39.829559 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f885b3a3-0db1-4e95-ad47-97520a7de897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:39.831557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1358/2709] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:52:39 2022] DELETE /identity/v3/projects/4a6ca476be674ec4a96993f135f66a80 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:52:39.837476 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719efcea-f612-4347-81ee-b57ce2f3a752 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:39.837756 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719efcea-f612-4347-81ee-b57ce2f3a752 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:39.838221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719efcea-f612-4347-81ee-b57ce2f3a752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:39.856368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-719efcea-f612-4347-81ee-b57ce2f3a752 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:52:39.857595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-719efcea-f612-4347-81ee-b57ce2f3a752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:52:39.916612 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1352/2710] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:52:39.924693 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a579a0-9e74-4fe4-9805-403320aeb170 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:39.924981 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a579a0-9e74-4fe4-9805-403320aeb170 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:39.925271 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a579a0-9e74-4fe4-9805-403320aeb170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:39.944005 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-41a579a0-9e74-4fe4-9805-403320aeb170 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:52:39.945227 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-41a579a0-9e74-4fe4-9805-403320aeb170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:52:39.953717 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1359/2711] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:52:39.958103 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719efcea-f612-4347-81ee-b57ce2f3a752 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:52:39.968288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719efcea-f612-4347-81ee-b57ce2f3a752 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:52:39.973292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99968c9-c414-42dc-b56b-21bfd2102e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:52:39.973959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99968c9-c414-42dc-b56b-21bfd2102e3b admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:39.974247 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99968c9-c414-42dc-b56b-21bfd2102e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:39.974525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99968c9-c414-42dc-b56b-21bfd2102e3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:39.975302 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99968c9-c414-42dc-b56b-21bfd2102e3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:52:39.975615 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99968c9-c414-42dc-b56b-21bfd2102e3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:52:39.977230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99968c9-c414-42dc-b56b-21bfd2102e3b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:52:39.985972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:52:39.989593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1353/2712] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 05:52:39.994252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a579a0-9e74-4fe4-9805-403320aeb170 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:52:40.004138 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a579a0-9e74-4fe4-9805-403320aeb170 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:52:40.009150 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b79f07-ddca-4f4b-9dfd-6b9fe024f014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:52:40.009804 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b79f07-ddca-4f4b-9dfd-6b9fe024f014 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:40.010093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b79f07-ddca-4f4b-9dfd-6b9fe024f014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:40.010367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b79f07-ddca-4f4b-9dfd-6b9fe024f014 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:40.011093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b79f07-ddca-4f4b-9dfd-6b9fe024f014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:52:40.013023 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b79f07-ddca-4f4b-9dfd-6b9fe024f014 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:52:40.042607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1360/2713] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:52:39 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:52:40.046932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99968c9-c414-42dc-b56b-21bfd2102e3b admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:52:40.057464 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99968c9-c414-42dc-b56b-21bfd2102e3b admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:52:40.116471 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15821d-03fd-4b5b-a5a5-885a2a6b05ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:52:40.117138 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15821d-03fd-4b5b-a5a5-885a2a6b05ec admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:40.117440 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15821d-03fd-4b5b-a5a5-885a2a6b05ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:40.117722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15821d-03fd-4b5b-a5a5-885a2a6b05ec admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:40.118415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15821d-03fd-4b5b-a5a5-885a2a6b05ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:52:40.120397 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15821d-03fd-4b5b-a5a5-885a2a6b05ec admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:52:40.142361 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1354/2714] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:52:40 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:52:40.146820 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b79f07-ddca-4f4b-9dfd-6b9fe024f014 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:52:40.158589 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b79f07-ddca-4f4b-9dfd-6b9fe024f014 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:52:40.164102 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe9363-43d5-4304-be7b-0b446ecf1d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:52:40.164777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe9363-43d5-4304-be7b-0b446ecf1d31 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:40.165061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe9363-43d5-4304-be7b-0b446ecf1d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:40.165341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe9363-43d5-4304-be7b-0b446ecf1d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:40.166107 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe9363-43d5-4304-be7b-0b446ecf1d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:52:40.167801 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe9363-43d5-4304-be7b-0b446ecf1d31 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:52:40.178557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1361/2715] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:52:40.182819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15821d-03fd-4b5b-a5a5-885a2a6b05ec admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:52:40.194061 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15821d-03fd-4b5b-a5a5-885a2a6b05ec admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:52:40.199763 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea6f33-6ebb-4339-8b8e-b30527b5e030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:52:40.200447 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea6f33-6ebb-4339-8b8e-b30527b5e030 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:40.200730 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea6f33-6ebb-4339-8b8e-b30527b5e030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:40.201008 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea6f33-6ebb-4339-8b8e-b30527b5e030 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:40.201752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea6f33-6ebb-4339-8b8e-b30527b5e030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:52:40.203549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea6f33-6ebb-4339-8b8e-b30527b5e030 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:52:40.214564 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1355/2716] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:52:40.220079 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe9363-43d5-4304-be7b-0b446ecf1d31 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:52:40.230500 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe9363-43d5-4304-be7b-0b446ecf1d31 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:52:40.235632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb5609-ff17-4bff-85c8-bbd7a52a047a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:52:40.236518 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb5609-ff17-4bff-85c8-bbd7a52a047a admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:40.236798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb5609-ff17-4bff-85c8-bbd7a52a047a admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:40.237184 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb5609-ff17-4bff-85c8-bbd7a52a047a admin admin] PATH_INFO: `/v3/projects/6237c768def644d1b514aa37b6ee2dfa/users/09d4361e3d794ceb8b947f35bced648f/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:40.252696 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb5609-ff17-4bff-85c8-bbd7a52a047a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6237c768def644d1b514aa37b6ee2dfa, user_id=09d4361e3d794ceb8b947f35bced648f, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:52:40.255553 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb5609-ff17-4bff-85c8-bbd7a52a047a admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:52:40.270301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1362/2717] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:52:40 2022] PUT /identity/v3/projects/6237c768def644d1b514aa37b6ee2dfa/users/09d4361e3d794ceb8b947f35bced648f/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:52:40.281983 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c48716-2261-496a-9567-c77914141226 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:40.282270 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c48716-2261-496a-9567-c77914141226 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:40.282541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c48716-2261-496a-9567-c77914141226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:40.311505 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-91c48716-2261-496a-9567-c77914141226 None None] MFA Rules not processed for user `09d4361e3d794ceb8b947f35bced648f`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:52:40.312687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-91c48716-2261-496a-9567-c77914141226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:52:40.379868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1356/2718] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:52:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:52:40.391508 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb5609-ff17-4bff-85c8-bbd7a52a047a admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:52:40.393363 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2cfb5609-ff17-4bff-85c8-bbd7a52a047a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:52:40.411526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:52:40.475649 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb5609-ff17-4bff-85c8-bbd7a52a047a admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:52:40.480478 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de24ab-9db8-4939-bbc8-fdfe38a16d13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:52:40.481167 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de24ab-9db8-4939-bbc8-fdfe38a16d13 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:40.481455 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de24ab-9db8-4939-bbc8-fdfe38a16d13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:40.481702 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de24ab-9db8-4939-bbc8-fdfe38a16d13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:40.491321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de24ab-9db8-4939-bbc8-fdfe38a16d13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:52:40.492762 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40de24ab-9db8-4939-bbc8-fdfe38a16d13 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:52:40.502482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1363/2719] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:52:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:52:47.928649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c48716-2261-496a-9567-c77914141226 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:52:47.930153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-91c48716-2261-496a-9567-c77914141226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:52:48.004847 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:52:48.007873 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c48716-2261-496a-9567-c77914141226 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:52:48.012383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c735cb-2e6c-4280-bb80-cf89c0fc46ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:52:48.013215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c735cb-2e6c-4280-bb80-cf89c0fc46ab service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:48.013657 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c735cb-2e6c-4280-bb80-cf89c0fc46ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:48.013939 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c735cb-2e6c-4280-bb80-cf89c0fc46ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:48.016457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-90c735cb-2e6c-4280-bb80-cf89c0fc46ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:52:48.087995 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c735cb-2e6c-4280-bb80-cf89c0fc46ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:52:48.089469 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c735cb-2e6c-4280-bb80-cf89c0fc46ab service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:52:48.100002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1357/2720] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:52:47 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:52:49.636912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de24ab-9db8-4939-bbc8-fdfe38a16d13 service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:52:49.712506 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de24ab-9db8-4939-bbc8-fdfe38a16d13 service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:52:49.717487 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c00101-7037-4f58-8960-d0477b6dd7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:52:49.718415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c00101-7037-4f58-8960-d0477b6dd7c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:49.718698 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c00101-7037-4f58-8960-d0477b6dd7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:49.718974 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c00101-7037-4f58-8960-d0477b6dd7c2 admin admin] PATH_INFO: `/v3/users/09d4361e3d794ceb8b947f35bced648f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:49.720266 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c00101-7037-4f58-8960-d0477b6dd7c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09d4361e3d794ceb8b947f35bced648f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:52:49.722285 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c00101-7037-4f58-8960-d0477b6dd7c2 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:52:49.779793 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-66c00101-7037-4f58-8960-d0477b6dd7c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09d4361e3d794ceb8b947f35bced648f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:49.807050 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-66c00101-7037-4f58-8960-d0477b6dd7c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09d4361e3d794ceb8b947f35bced648f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:49.818851 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-66c00101-7037-4f58-8960-d0477b6dd7c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09d4361e3d794ceb8b947f35bced648f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:49.819687 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-66c00101-7037-4f58-8960-d0477b6dd7c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09d4361e3d794ceb8b947f35bced648f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:49.820329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-66c00101-7037-4f58-8960-d0477b6dd7c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09d4361e3d794ceb8b947f35bced648f'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:49.834608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1364/2721] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:52:49 2022] DELETE /identity/v3/users/09d4361e3d794ceb8b947f35bced648f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:52:49.839363 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc142d2-7fb3-4294-a9cb-c1eaebe30627 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:49.839834 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc142d2-7fb3-4294-a9cb-c1eaebe30627 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:49.839964 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc142d2-7fb3-4294-a9cb-c1eaebe30627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:49.857909 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-6cc142d2-7fb3-4294-a9cb-c1eaebe30627 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:52:49.859233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-6cc142d2-7fb3-4294-a9cb-c1eaebe30627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:52:49.923273 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1358/2722] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:52:49.933095 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c00101-7037-4f58-8960-d0477b6dd7c2 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:52:49.935866 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-66c00101-7037-4f58-8960-d0477b6dd7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:52:49.961381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:52:50.016219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c00101-7037-4f58-8960-d0477b6dd7c2 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:52:50.020050 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62f52c-01f2-4243-a5b2-d4c1b3481c01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:52:50.021124 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62f52c-01f2-4243-a5b2-d4c1b3481c01 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:50.021124 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62f52c-01f2-4243-a5b2-d4c1b3481c01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:50.021320 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62f52c-01f2-4243-a5b2-d4c1b3481c01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:50.031481 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d62f52c-01f2-4243-a5b2-d4c1b3481c01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:52:50.032874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d62f52c-01f2-4243-a5b2-d4c1b3481c01 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:52:50.042760 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1365/2723] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:52:50.442250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc142d2-7fb3-4294-a9cb-c1eaebe30627 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:52:50.443951 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-6cc142d2-7fb3-4294-a9cb-c1eaebe30627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:52:50.456732 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc142d2-7fb3-4294-a9cb-c1eaebe30627 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:52:50.462416 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:52:50.463347 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:52:50.463627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:52:50.464063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b admin admin] PATH_INFO: `/v3/projects/6237c768def644d1b514aa37b6ee2dfa` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:52:50.466129 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6237c768def644d1b514aa37b6ee2dfa)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:52:50.468493 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:52:50.524468 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6237c768def644d1b514aa37b6ee2dfa', 'request_id': 'req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:50.531899 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6237c768def644d1b514aa37b6ee2dfa', 'request_id': 'req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:50.538564 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6237c768def644d1b514aa37b6ee2dfa', 'request_id': 'req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:50.553377 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b admin admin] The token cache is being invalidate because project 6237c768def644d1b514aa37b6ee2dfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:52:50.554114 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:50.554719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:52:50.557016 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1359/2724] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:52:50 2022] DELETE /identity/v3/projects/6237c768def644d1b514aa37b6ee2dfa => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:53:00.038461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=19 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:53:00.550184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:53:09.601819 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd9cab-0e50-4a67-8d47-4b4a103c5dd5 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:53:09.602332 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd9cab-0e50-4a67-8d47-4b4a103c5dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:53:09.602332 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd9cab-0e50-4a67-8d47-4b4a103c5dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:53:09.620533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-71bd9cab-0e50-4a67-8d47-4b4a103c5dd5 None None] MFA Rules not processed for user `17108265cfba44b29c362ac18bbf3e20`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:53:09.621766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-71bd9cab-0e50-4a67-8d47-4b4a103c5dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:53:09.806901 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1366/2725] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:53:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:53:19.800345 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=25 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:53:40.382211 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:53:40.383822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:53:40.441849 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d748c48-2f06-4c9f-a31a-fad9571b4c6b admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:53:40.445734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18069cab-787b-4eda-8e1c-153655db1732 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:53:40.446390 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18069cab-787b-4eda-8e1c-153655db1732 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:53:40.446675 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18069cab-787b-4eda-8e1c-153655db1732 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:53:40.446954 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18069cab-787b-4eda-8e1c-153655db1732 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:53:40.448892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-18069cab-787b-4eda-8e1c-153655db1732 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:53:40.520089 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18069cab-787b-4eda-8e1c-153655db1732 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:53:40.521535 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18069cab-787b-4eda-8e1c-153655db1732 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:53:40.533185 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1360/2726] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:53:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:53:43.819460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf83b78-15f6-4cad-ba36-6856d5de1fc6 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:53:43.820012 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf83b78-15f6-4cad-ba36-6856d5de1fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:53:43.820012 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf83b78-15f6-4cad-ba36-6856d5de1fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:53:43.841246 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-bbf83b78-15f6-4cad-ba36-6856d5de1fc6 None None] MFA Rules not processed for user `4d04998b636649d98103b1cd5f3a0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:53:43.842465 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-bbf83b78-15f6-4cad-ba36-6856d5de1fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:53:43.903789 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1367/2727] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jul 21 05:53:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:53:43.952030 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18069cab-787b-4eda-8e1c-153655db1732 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:53:43.953540 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-18069cab-787b-4eda-8e1c-153655db1732 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:53:44.036429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18069cab-787b-4eda-8e1c-153655db1732 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:53:44.041811 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fb710-2580-4294-b077-8096a25c7742 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61b61188b5b94a51ba1ae1a88b78b89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:53:44.042465 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fb710-2580-4294-b077-8096a25c7742 service swift] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:53:44.042780 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fb710-2580-4294-b077-8096a25c7742 service swift] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:53:44.043065 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fb710-2580-4294-b077-8096a25c7742 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:53:44.051868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6fb710-2580-4294-b077-8096a25c7742 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:53:44.053454 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6fb710-2580-4294-b077-8096a25c7742 service swift] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:53:44.062108 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1361/2728] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jul 21 05:53:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 05:53:46.394673 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf83b78-15f6-4cad-ba36-6856d5de1fc6 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:53:46.397007 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-bbf83b78-15f6-4cad-ba36-6856d5de1fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:53:46.413920 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf83b78-15f6-4cad-ba36-6856d5de1fc6 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:53:46.419939 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876de283-1be6-4400-ab7c-9d9f22ff8ab3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:53:46.420943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876de283-1be6-4400-ab7c-9d9f22ff8ab3 service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:53:46.421484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876de283-1be6-4400-ab7c-9d9f22ff8ab3 service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:53:46.421942 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876de283-1be6-4400-ab7c-9d9f22ff8ab3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:53:46.424396 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-876de283-1be6-4400-ab7c-9d9f22ff8ab3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:53:46.508717 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876de283-1be6-4400-ab7c-9d9f22ff8ab3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:53:46.510333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876de283-1be6-4400-ab7c-9d9f22ff8ab3 service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:53:46.529630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1368/2729] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:53:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:53:54.058494 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=86 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:53:56.527092 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=66 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:53:58.248002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fee79f-fe7c-4658-ae8c-bc6c794d2a67 None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:53:58.248806 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fee79f-fe7c-4658-ae8c-bc6c794d2a67 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:53:58.249225 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fee79f-fe7c-4658-ae8c-bc6c794d2a67 None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:53:58.252033 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1362/2730] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jul 21 05:53:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:53:58.259329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be717d-794b-4488-b6d1-40b7490e3f87 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:53:58.259876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be717d-794b-4488-b6d1-40b7490e3f87 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:53:58.259876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be717d-794b-4488-b6d1-40b7490e3f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:53:58.298717 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-11be717d-794b-4488-b6d1-40b7490e3f87 None None] MFA Rules not processed for user `928d014648cd41d7b58e7e01845eb748`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:53:58.300198 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-11be717d-794b-4488-b6d1-40b7490e3f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:53:58.310217 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1369/2731] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jul 21 05:53:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:53:58.323107 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fee79f-fe7c-4658-ae8c-bc6c794d2a67 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:53:58.324807 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-52fee79f-fe7c-4658-ae8c-bc6c794d2a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:53:58.383119 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fee79f-fe7c-4658-ae8c-bc6c794d2a67 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:53:58.386552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39897562-ab82-473b-a2a4-907ef8bc14df service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2707c6967a0a419a98ae142ff1d1a961', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:53:58.387231 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39897562-ab82-473b-a2a4-907ef8bc14df service glance] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:53:58.387518 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39897562-ab82-473b-a2a4-907ef8bc14df service glance] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:53:58.387794 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39897562-ab82-473b-a2a4-907ef8bc14df service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:53:58.397732 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39897562-ab82-473b-a2a4-907ef8bc14df service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:53:58.399205 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39897562-ab82-473b-a2a4-907ef8bc14df service glance] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:53:58.410126 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1363/2732] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:53:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:53:58.484795 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960b857-f4ac-4b57-aa54-7da5f00d7900 None None] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:53:58.485101 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960b857-f4ac-4b57-aa54-7da5f00d7900 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:53:58.485373 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960b857-f4ac-4b57-aa54-7da5f00d7900 None None] PATH_INFO: `/` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:53:58.486753 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1370/2733] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jul 21 05:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:53:58.492754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98285f-efc4-4f77-9fda-64e559658e3d None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:53:58.493047 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98285f-efc4-4f77-9fda-64e559658e3d None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:53:58.493328 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98285f-efc4-4f77-9fda-64e559658e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:53:58.529671 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-cf98285f-efc4-4f77-9fda-64e559658e3d None None] MFA Rules not processed for user `e07508a9fa9146528620b7f8abec8549`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:53:58.531292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-cf98285f-efc4-4f77-9fda-64e559658e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:53:58.538556 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1364/2734] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jul 21 05:53:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:53:58.550651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960b857-f4ac-4b57-aa54-7da5f00d7900 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:53:58.563112 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960b857-f4ac-4b57-aa54-7da5f00d7900 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:53:58.566929 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b0891-902d-400e-af7e-bf65a1416a24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:53:58.567597 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b0891-902d-400e-af7e-bf65a1416a24 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:53:58.567875 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b0891-902d-400e-af7e-bf65a1416a24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:53:58.568152 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b0891-902d-400e-af7e-bf65a1416a24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:53:58.575255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b0891-902d-400e-af7e-bf65a1416a24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:53:58.576688 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b0891-902d-400e-af7e-bf65a1416a24 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:53:58.585119 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1371/2735] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:53:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:53:58.670204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783854f2-91ff-4504-8cee-ca12e544b60d None None] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:53:58.670492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783854f2-91ff-4504-8cee-ca12e544b60d None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:53:58.670789 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783854f2-91ff-4504-8cee-ca12e544b60d None None] PATH_INFO: `/` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:53:58.672215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1365/2736] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jul 21 05:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 21 05:53:58.677675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5758c0-fc4a-481c-81a2-5000403a5ab3 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:53:58.677964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5758c0-fc4a-481c-81a2-5000403a5ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:53:58.678234 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5758c0-fc4a-481c-81a2-5000403a5ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:53:58.697109 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-2e5758c0-fc4a-481c-81a2-5000403a5ab3 None None] MFA Rules not processed for user `928d014648cd41d7b58e7e01845eb748`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:53:58.698336 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2e5758c0-fc4a-481c-81a2-5000403a5ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:53:58.708028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1372/2737] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jul 21 05:53:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:53:58.719534 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783854f2-91ff-4504-8cee-ca12e544b60d None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:53:58.721258 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-783854f2-91ff-4504-8cee-ca12e544b60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:53:58.735160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783854f2-91ff-4504-8cee-ca12e544b60d None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:53:58.739185 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b3b77-61be-496b-bd4a-8dbaa99cbf2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:53:58.739826 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b3b77-61be-496b-bd4a-8dbaa99cbf2d service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:53:58.740108 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b3b77-61be-496b-bd4a-8dbaa99cbf2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:53:58.740382 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b3b77-61be-496b-bd4a-8dbaa99cbf2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:53:58.751267 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b3b77-61be-496b-bd4a-8dbaa99cbf2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:53:58.752718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b3b77-61be-496b-bd4a-8dbaa99cbf2d service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:53:58.763829 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1366/2738] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:53:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:54:00.478008 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329250d3-3f1c-4953-b890-52de491f5700 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:54:00.478502 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329250d3-3f1c-4953-b890-52de491f5700 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:54:00.478606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329250d3-3f1c-4953-b890-52de491f5700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:54:00.530313 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-329250d3-3f1c-4953-b890-52de491f5700 None None] MFA Rules not processed for user `4d04998b636649d98103b1cd5f3a0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:54:00.531921 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-329250d3-3f1c-4953-b890-52de491f5700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:54:00.550020 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1373/2739] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jul 21 05:54:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:54:00.562141 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b3b77-61be-496b-bd4a-8dbaa99cbf2d service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:54:00.577320 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b3b77-61be-496b-bd4a-8dbaa99cbf2d service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:54:00.584573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d46f5-054b-4701-bbd5-2bc97564f0e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61b61188b5b94a51ba1ae1a88b78b89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:54:00.585328 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d46f5-054b-4701-bbd5-2bc97564f0e5 service swift] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:54:00.585676 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d46f5-054b-4701-bbd5-2bc97564f0e5 service swift] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:54:00.586021 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d46f5-054b-4701-bbd5-2bc97564f0e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:54:00.600221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d46f5-054b-4701-bbd5-2bc97564f0e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:54:00.601829 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d46f5-054b-4701-bbd5-2bc97564f0e5 service swift] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:54:00.611760 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1367/2740] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jul 21 05:54:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 05:54:10.546873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=36 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:54:10.607905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=47 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:54:16.417074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329250d3-3f1c-4953-b890-52de491f5700 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:54:16.427606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329250d3-3f1c-4953-b890-52de491f5700 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:54:16.432025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d9e3c-194b-4cef-9d5f-549f46c7e89d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:54:16.432669 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d9e3c-194b-4cef-9d5f-549f46c7e89d service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:54:16.432951 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d9e3c-194b-4cef-9d5f-549f46c7e89d service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:54:16.433224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d9e3c-194b-4cef-9d5f-549f46c7e89d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:54:16.435095 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1a6d9e3c-194b-4cef-9d5f-549f46c7e89d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:54:16.445095 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d9e3c-194b-4cef-9d5f-549f46c7e89d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:54:16.446546 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d9e3c-194b-4cef-9d5f-549f46c7e89d service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:54:16.455065 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1374/2741] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:54:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:54:18.373985 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de2e27-b26b-4a90-b344-3f2ccb6eff86 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:54:18.374472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de2e27-b26b-4a90-b344-3f2ccb6eff86 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:54:18.374587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de2e27-b26b-4a90-b344-3f2ccb6eff86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:54:18.393282 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-69de2e27-b26b-4a90-b344-3f2ccb6eff86 None None] MFA Rules not processed for user `4d04998b636649d98103b1cd5f3a0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:54:18.394470 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-69de2e27-b26b-4a90-b344-3f2ccb6eff86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:54:18.402488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1368/2742] 10.222.0.44 () {58 vars in 967 bytes} [Thu Jul 21 05:54:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:54:18.415732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d9e3c-194b-4cef-9d5f-549f46c7e89d service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:54:18.443095 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d9e3c-194b-4cef-9d5f-549f46c7e89d service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:54:18.448740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae966bb8-74f1-48ba-b785-1e0fce07949e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61b61188b5b94a51ba1ae1a88b78b89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:54:18.449395 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae966bb8-74f1-48ba-b785-1e0fce07949e service swift] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:54:18.449692 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae966bb8-74f1-48ba-b785-1e0fce07949e service swift] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:54:18.449956 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae966bb8-74f1-48ba-b785-1e0fce07949e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:54:18.458655 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae966bb8-74f1-48ba-b785-1e0fce07949e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:54:18.460353 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae966bb8-74f1-48ba-b785-1e0fce07949e service swift] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:54:18.468642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1375/2743] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jul 21 05:54:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 05:54:28.388734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:54:28.465411 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=16 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:55:18.729322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de2e27-b26b-4a90-b344-3f2ccb6eff86 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:18.730833 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-69de2e27-b26b-4a90-b344-3f2ccb6eff86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:18.808798 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de2e27-b26b-4a90-b344-3f2ccb6eff86 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:18.815586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbbf64-7a78-404c-a1e9-f6abd5dd44aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:18.816298 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbbf64-7a78-404c-a1e9-f6abd5dd44aa service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:18.816717 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbbf64-7a78-404c-a1e9-f6abd5dd44aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:18.817009 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbbf64-7a78-404c-a1e9-f6abd5dd44aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:18.820157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-2acbbf64-7a78-404c-a1e9-f6abd5dd44aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:18.835285 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbbf64-7a78-404c-a1e9-f6abd5dd44aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:18.836251 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae966bb8-74f1-48ba-b785-1e0fce07949e service swift] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:18.837550 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbbf64-7a78-404c-a1e9-f6abd5dd44aa service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:18.850889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1369/2744] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:55:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:55:18.851849 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae966bb8-74f1-48ba-b785-1e0fce07949e service swift] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:18.857687 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a482263-05cb-45bc-a4d5-ba5e10c9aa4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:18.858426 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a482263-05cb-45bc-a4d5-ba5e10c9aa4d service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:18.858851 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a482263-05cb-45bc-a4d5-ba5e10c9aa4d service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:18.859228 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a482263-05cb-45bc-a4d5-ba5e10c9aa4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:18.861418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2a482263-05cb-45bc-a4d5-ba5e10c9aa4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:18.875081 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a482263-05cb-45bc-a4d5-ba5e10c9aa4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:18.876779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a482263-05cb-45bc-a4d5-ba5e10c9aa4d service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:18.890755 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1376/2745] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:55:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:55:24.873381 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbbf64-7a78-404c-a1e9-f6abd5dd44aa service cinder] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:24.883789 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbbf64-7a78-404c-a1e9-f6abd5dd44aa service cinder] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:24.887518 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8cafb-1c8b-4675-af94-6528fbf5deeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:24.888276 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8cafb-1c8b-4675-af94-6528fbf5deeb service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:24.888623 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8cafb-1c8b-4675-af94-6528fbf5deeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:24.888967 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8cafb-1c8b-4675-af94-6528fbf5deeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:24.890791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8ec8cafb-1c8b-4675-af94-6528fbf5deeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:24.901497 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8cafb-1c8b-4675-af94-6528fbf5deeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:24.902959 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8cafb-1c8b-4675-af94-6528fbf5deeb service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:24.911846 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1370/2746] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:55:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:55:25.010935 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a482263-05cb-45bc-a4d5-ba5e10c9aa4d service nova] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:25.021147 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a482263-05cb-45bc-a4d5-ba5e10c9aa4d service nova] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:25.025217 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36472c-05b1-4935-91df-e775e09cbeef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:25.025873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36472c-05b1-4935-91df-e775e09cbeef service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:25.026151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36472c-05b1-4935-91df-e775e09cbeef service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:25.026420 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36472c-05b1-4935-91df-e775e09cbeef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:25.028324 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4a36472c-05b1-4935-91df-e775e09cbeef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:25.043508 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a36472c-05b1-4935-91df-e775e09cbeef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:25.044967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a36472c-05b1-4935-91df-e775e09cbeef service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:25.056326 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1377/2747] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:55:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:55:25.119573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8cafb-1c8b-4675-af94-6528fbf5deeb service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:25.121092 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-8ec8cafb-1c8b-4675-af94-6528fbf5deeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:25.195135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8cafb-1c8b-4675-af94-6528fbf5deeb service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:25.200634 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc9e67-f54e-422f-861b-e6003f30e11b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:25.201277 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc9e67-f54e-422f-861b-e6003f30e11b service placement] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:25.201565 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc9e67-f54e-422f-861b-e6003f30e11b service placement] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:25.201840 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc9e67-f54e-422f-861b-e6003f30e11b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:25.204256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-73fc9e67-f54e-422f-861b-e6003f30e11b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:25.216154 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc9e67-f54e-422f-861b-e6003f30e11b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:25.217767 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc9e67-f54e-422f-861b-e6003f30e11b service placement] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:25.232665 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1371/2748] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:55:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:55:25.426620 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36472c-05b1-4935-91df-e775e09cbeef service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:25.435581 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36472c-05b1-4935-91df-e775e09cbeef service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:25.439323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc286504-15b1-4cd5-8ec2-ab1dbce7302f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:25.439971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc286504-15b1-4cd5-8ec2-ab1dbce7302f service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:25.440258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc286504-15b1-4cd5-8ec2-ab1dbce7302f service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:25.440530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc286504-15b1-4cd5-8ec2-ab1dbce7302f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:25.442371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-fc286504-15b1-4cd5-8ec2-ab1dbce7302f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:25.511812 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc286504-15b1-4cd5-8ec2-ab1dbce7302f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:25.513259 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc286504-15b1-4cd5-8ec2-ab1dbce7302f service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:25.524676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1378/2749] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:55:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:55:28.801421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc9e67-f54e-422f-861b-e6003f30e11b service placement] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:28.801421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-73fc9e67-f54e-422f-861b-e6003f30e11b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:28.813398 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc9e67-f54e-422f-861b-e6003f30e11b service placement] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:28.817884 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57667de-e3b4-44d3-822f-0419942b0baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:28.818759 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57667de-e3b4-44d3-822f-0419942b0baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:28.819066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57667de-e3b4-44d3-822f-0419942b0baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:28.819345 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57667de-e3b4-44d3-822f-0419942b0baf admin admin] PATH_INFO: `/v3/users/0cbf797171144c139c72c516cea59bb9` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:28.820529 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57667de-e3b4-44d3-822f-0419942b0baf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cbf797171144c139c72c516cea59bb9)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:28.822441 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57667de-e3b4-44d3-822f-0419942b0baf admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:28.869257 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-c57667de-e3b4-44d3-822f-0419942b0baf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cbf797171144c139c72c516cea59bb9'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:55:28.880973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-c57667de-e3b4-44d3-822f-0419942b0baf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cbf797171144c139c72c516cea59bb9'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:55:28.887811 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-c57667de-e3b4-44d3-822f-0419942b0baf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cbf797171144c139c72c516cea59bb9'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:55:28.888535 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-c57667de-e3b4-44d3-822f-0419942b0baf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cbf797171144c139c72c516cea59bb9'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:55:28.889154 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-c57667de-e3b4-44d3-822f-0419942b0baf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cbf797171144c139c72c516cea59bb9'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:55:28.890835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:55:28.902653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1372/2750] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:55:28 2022] DELETE /identity/v3/users/0cbf797171144c139c72c516cea59bb9 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:55:29.077355 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc286504-15b1-4cd5-8ec2-ab1dbce7302f service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:29.089652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc286504-15b1-4cd5-8ec2-ab1dbce7302f service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:29.148938 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9492054-d142-436b-a73d-c04d5e8b7cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:29.149806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9492054-d142-436b-a73d-c04d5e8b7cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:29.150090 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9492054-d142-436b-a73d-c04d5e8b7cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:29.150359 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9492054-d142-436b-a73d-c04d5e8b7cbf admin admin] PATH_INFO: `/v3/projects/7b88b7408762493e825c86d05b5e62bc` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:29.151708 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9492054-d142-436b-a73d-c04d5e8b7cbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b88b7408762493e825c86d05b5e62bc)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:29.153628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9492054-d142-436b-a73d-c04d5e8b7cbf admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:29.201720 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f9492054-d142-436b-a73d-c04d5e8b7cbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b88b7408762493e825c86d05b5e62bc', 'request_id': 'req-f9492054-d142-436b-a73d-c04d5e8b7cbf'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:55:29.207115 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f9492054-d142-436b-a73d-c04d5e8b7cbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b88b7408762493e825c86d05b5e62bc', 'request_id': 'req-f9492054-d142-436b-a73d-c04d5e8b7cbf'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:55:29.215107 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f9492054-d142-436b-a73d-c04d5e8b7cbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b88b7408762493e825c86d05b5e62bc', 'request_id': 'req-f9492054-d142-436b-a73d-c04d5e8b7cbf'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:55:29.222074 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:55:29.227138 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f9492054-d142-436b-a73d-c04d5e8b7cbf admin admin] The token cache is being invalidate because project 7b88b7408762493e825c86d05b5e62bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:55:29.227430 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f9492054-d142-436b-a73d-c04d5e8b7cbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:55:29.228324 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-f9492054-d142-436b-a73d-c04d5e8b7cbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:55:29.230720 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1379/2751] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:55:29 2022] DELETE /identity/v3/projects/7b88b7408762493e825c86d05b5e62bc => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:55:29.236596 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3d687-cb4f-4604-9fef-451373c75eb7 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:29.237051 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3d687-cb4f-4604-9fef-451373c75eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:29.237685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3d687-cb4f-4604-9fef-451373c75eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:29.257740 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-03c3d687-cb4f-4604-9fef-451373c75eb7 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:55:29.258921 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-03c3d687-cb4f-4604-9fef-451373c75eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:29.321822 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1373/2752] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:55:29.329874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f8203-8714-493f-8c9f-2c51a1e3d313 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:29.330162 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f8203-8714-493f-8c9f-2c51a1e3d313 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:29.330435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f8203-8714-493f-8c9f-2c51a1e3d313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:29.349354 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-823f8203-8714-493f-8c9f-2c51a1e3d313 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:55:29.350558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-823f8203-8714-493f-8c9f-2c51a1e3d313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:29.360304 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1380/2753] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:55:29.365129 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3d687-cb4f-4604-9fef-451373c75eb7 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:29.375474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3d687-cb4f-4604-9fef-451373c75eb7 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:29.381474 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19836d2-f1ec-44c5-8f82-860d8cc504c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:29.382132 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19836d2-f1ec-44c5-8f82-860d8cc504c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:29.382410 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19836d2-f1ec-44c5-8f82-860d8cc504c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:29.382681 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19836d2-f1ec-44c5-8f82-860d8cc504c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:29.383441 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19836d2-f1ec-44c5-8f82-860d8cc504c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 05:55:29.383746 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19836d2-f1ec-44c5-8f82-860d8cc504c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:29.385399 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19836d2-f1ec-44c5-8f82-860d8cc504c2 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:29.397759 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1374/2754] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 05:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 05:55:29.402212 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f8203-8714-493f-8c9f-2c51a1e3d313 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:29.412263 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f8203-8714-493f-8c9f-2c51a1e3d313 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:29.418515 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8c0bf-ba4c-405f-b084-0905d1ef9702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:29.419193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8c0bf-ba4c-405f-b084-0905d1ef9702 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:29.419482 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8c0bf-ba4c-405f-b084-0905d1ef9702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:29.419758 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de8c0bf-ba4c-405f-b084-0905d1ef9702 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:29.420451 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8c0bf-ba4c-405f-b084-0905d1ef9702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:29.422360 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de8c0bf-ba4c-405f-b084-0905d1ef9702 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:29.460948 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1381/2755] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 05:55:29 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 05:55:29.465135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19836d2-f1ec-44c5-8f82-860d8cc504c2 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:29.475690 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19836d2-f1ec-44c5-8f82-860d8cc504c2 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:29.544927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74effbd6-9fe0-4357-a7d8-aae1340012ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:29.545609 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74effbd6-9fe0-4357-a7d8-aae1340012ba admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:29.545893 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74effbd6-9fe0-4357-a7d8-aae1340012ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:29.546200 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74effbd6-9fe0-4357-a7d8-aae1340012ba admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:29.546900 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74effbd6-9fe0-4357-a7d8-aae1340012ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:29.549036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74effbd6-9fe0-4357-a7d8-aae1340012ba admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:29.570204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1375/2756] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 05:55:29 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 05:55:29.574929 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8c0bf-ba4c-405f-b084-0905d1ef9702 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:29.587149 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de8c0bf-ba4c-405f-b084-0905d1ef9702 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:29.593137 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb67659-d2b3-4eb2-9163-8067856ae3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:29.593826 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb67659-d2b3-4eb2-9163-8067856ae3b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:29.594154 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb67659-d2b3-4eb2-9163-8067856ae3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:29.594460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb67659-d2b3-4eb2-9163-8067856ae3b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:29.595318 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb67659-d2b3-4eb2-9163-8067856ae3b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:29.597068 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb67659-d2b3-4eb2-9163-8067856ae3b5 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:29.610400 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1382/2757] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:55:29.614421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74effbd6-9fe0-4357-a7d8-aae1340012ba admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:29.625316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74effbd6-9fe0-4357-a7d8-aae1340012ba admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:29.631319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780f4e9-fe2e-41cd-a42b-c7fc436c13ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:29.631999 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780f4e9-fe2e-41cd-a42b-c7fc436c13ba admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:29.632283 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780f4e9-fe2e-41cd-a42b-c7fc436c13ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:29.632555 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780f4e9-fe2e-41cd-a42b-c7fc436c13ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:29.633279 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780f4e9-fe2e-41cd-a42b-c7fc436c13ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:29.634896 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780f4e9-fe2e-41cd-a42b-c7fc436c13ba admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:29.646392 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1376/2758] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 05:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 05:55:29.651860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb67659-d2b3-4eb2-9163-8067856ae3b5 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:29.664639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb67659-d2b3-4eb2-9163-8067856ae3b5 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:29.672694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89ce8b-bf72-4133-8f0f-537879f151b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:29.673757 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89ce8b-bf72-4133-8f0f-537879f151b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:29.674071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89ce8b-bf72-4133-8f0f-537879f151b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:29.674351 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89ce8b-bf72-4133-8f0f-537879f151b2 admin admin] PATH_INFO: `/v3/projects/a3f30d9c29794ccfb52188216cd48085/users/e5a03f7524f7436fbbd78a7778702d1b/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:29.690762 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89ce8b-bf72-4133-8f0f-537879f151b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3f30d9c29794ccfb52188216cd48085, user_id=e5a03f7524f7436fbbd78a7778702d1b, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:29.693555 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89ce8b-bf72-4133-8f0f-537879f151b2 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:29.711677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1383/2759] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 05:55:29 2022] PUT /identity/v3/projects/a3f30d9c29794ccfb52188216cd48085/users/e5a03f7524f7436fbbd78a7778702d1b/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 05:55:29.717120 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ebb58-6e4e-426f-beeb-b0cc881e1d38 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:29.717407 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ebb58-6e4e-426f-beeb-b0cc881e1d38 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:29.717684 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ebb58-6e4e-426f-beeb-b0cc881e1d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:29.736339 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-f50ebb58-6e4e-426f-beeb-b0cc881e1d38 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:55:29.737651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-f50ebb58-6e4e-426f-beeb-b0cc881e1d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:29.789647 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:55:29.801466 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1377/2760] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:55:29.814875 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89ce8b-bf72-4133-8f0f-537879f151b2 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:29.816787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-5e89ce8b-bf72-4133-8f0f-537879f151b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:29.881224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89ce8b-bf72-4133-8f0f-537879f151b2 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:29.886962 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be612691-5cde-44d4-9984-cade8ac0c3d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:29.887653 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be612691-5cde-44d4-9984-cade8ac0c3d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:29.887935 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be612691-5cde-44d4-9984-cade8ac0c3d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:29.888323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be612691-5cde-44d4-9984-cade8ac0c3d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:29.901610 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be612691-5cde-44d4-9984-cade8ac0c3d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:29.903556 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be612691-5cde-44d4-9984-cade8ac0c3d5 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:29.917028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1384/2761] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:55:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:55:29.924927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e3866-f610-436f-916a-f90fc32616f1 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:29.925221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e3866-f610-436f-916a-f90fc32616f1 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:29.925508 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e3866-f610-436f-916a-f90fc32616f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:29.943819 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-321e3866-f610-436f-916a-f90fc32616f1 None None] MFA Rules not processed for user `17108265cfba44b29c362ac18bbf3e20`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:55:29.945044 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-321e3866-f610-436f-916a-f90fc32616f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:30.009202 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1378/2762] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:55:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 05:55:34.749042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38139d48-9c07-413b-8024-dff0d531aa69 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:34.749533 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38139d48-9c07-413b-8024-dff0d531aa69 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:34.749718 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38139d48-9c07-413b-8024-dff0d531aa69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:34.783591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-38139d48-9c07-413b-8024-dff0d531aa69 None None] MFA Rules not processed for user `e5a03f7524f7436fbbd78a7778702d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:55:34.785015 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-38139d48-9c07-413b-8024-dff0d531aa69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:34.892624 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1385/2763] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:55:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:55:34.907696 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e3866-f610-436f-916a-f90fc32616f1 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:34.909382 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-321e3866-f610-436f-916a-f90fc32616f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:35.024473 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e3866-f610-436f-916a-f90fc32616f1 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:35.033218 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f0a1f-3620-4e91-b184-f324b04e2ae4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:35.034254 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f0a1f-3620-4e91-b184-f324b04e2ae4 service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:35.034705 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f0a1f-3620-4e91-b184-f324b04e2ae4 service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:35.035135 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f0a1f-3620-4e91-b184-f324b04e2ae4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:35.046800 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f0a1f-3620-4e91-b184-f324b04e2ae4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:35.050590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f0a1f-3620-4e91-b184-f324b04e2ae4 service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:35.061814 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1379/2764] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:55:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:55:36.706674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9430a-6e18-48ef-a230-9cebee12554e None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:36.707309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9430a-6e18-48ef-a230-9cebee12554e None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:36.707309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9430a-6e18-48ef-a230-9cebee12554e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:36.726580 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-8ae9430a-6e18-48ef-a230-9cebee12554e None None] MFA Rules not processed for user `e5a03f7524f7436fbbd78a7778702d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 05:55:36.727831 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8ae9430a-6e18-48ef-a230-9cebee12554e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:36.734742 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1386/2765] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 05:55:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 05:55:41.019754 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f0a1f-3620-4e91-b184-f324b04e2ae4 service nova] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:41.033938 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f0a1f-3620-4e91-b184-f324b04e2ae4 service nova] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:41.041453 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952279f-87e2-4b4b-93c5-bad1260ea568 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:41.042313 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952279f-87e2-4b4b-93c5-bad1260ea568 service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:41.042711 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952279f-87e2-4b4b-93c5-bad1260ea568 service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:41.043133 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952279f-87e2-4b4b-93c5-bad1260ea568 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:41.045396 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0952279f-87e2-4b4b-93c5-bad1260ea568 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:41.060310 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952279f-87e2-4b4b-93c5-bad1260ea568 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:41.062112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952279f-87e2-4b4b-93c5-bad1260ea568 service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:41.074599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1380/2766] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:55:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:55:46.721423 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=93,keystone:INSERT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:55:51.070138 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=66 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:55:51.341747 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9430a-6e18-48ef-a230-9cebee12554e None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:55:51.352454 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9430a-6e18-48ef-a230-9cebee12554e None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:55:51.356450 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86152172-8c06-4371-b2a6-8c1f0df08c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:55:51.357097 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86152172-8c06-4371-b2a6-8c1f0df08c88 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:55:51.357384 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86152172-8c06-4371-b2a6-8c1f0df08c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:55:51.357656 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86152172-8c06-4371-b2a6-8c1f0df08c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:55:51.359484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-86152172-8c06-4371-b2a6-8c1f0df08c88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:55:51.434721 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86152172-8c06-4371-b2a6-8c1f0df08c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:55:51.436136 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86152172-8c06-4371-b2a6-8c1f0df08c88 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:55:51.446328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1387/2767] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:55:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:56:01.441625 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=30 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:56:54.234444 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952279f-87e2-4b4b-93c5-bad1260ea568 service nova] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:56:54.236187 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-0952279f-87e2-4b4b-93c5-bad1260ea568 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:56:54.309004 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952279f-87e2-4b4b-93c5-bad1260ea568 service nova] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:56:54.314728 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94774469-503a-4591-9bb3-d9e22bf0e51d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:56:54.315411 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94774469-503a-4591-9bb3-d9e22bf0e51d service placement] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:56:54.315696 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94774469-503a-4591-9bb3-d9e22bf0e51d service placement] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:56:54.315973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94774469-503a-4591-9bb3-d9e22bf0e51d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:56:54.318081 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-94774469-503a-4591-9bb3-d9e22bf0e51d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:56:54.329598 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94774469-503a-4591-9bb3-d9e22bf0e51d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:56:54.331243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94774469-503a-4591-9bb3-d9e22bf0e51d service placement] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:56:54.341060 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1381/2768] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:56:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:57:04.337344 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=32 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:57:32.840953 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86152172-8c06-4371-b2a6-8c1f0df08c88 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:57:32.843660 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-86152172-8c06-4371-b2a6-8c1f0df08c88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:57:32.855603 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86152172-8c06-4371-b2a6-8c1f0df08c88 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:57:32.860471 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15c5c9-5951-4a39-be7f-7bf658cf4997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:57:32.861045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15c5c9-5951-4a39-be7f-7bf658cf4997 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:57:32.861314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15c5c9-5951-4a39-be7f-7bf658cf4997 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:57:32.861591 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb15c5c9-5951-4a39-be7f-7bf658cf4997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:57:32.863615 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-cb15c5c9-5951-4a39-be7f-7bf658cf4997 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:57:32.929644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15c5c9-5951-4a39-be7f-7bf658cf4997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:57:32.931144 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb15c5c9-5951-4a39-be7f-7bf658cf4997 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:57:32.940716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1388/2769] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:57:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 05:57:34.063216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94774469-503a-4591-9bb3-d9e22bf0e51d service placement] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:57:34.076627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94774469-503a-4591-9bb3-d9e22bf0e51d service placement] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:57:34.082526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167631b-d10f-474f-8626-30afe8826b21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:57:34.083217 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167631b-d10f-474f-8626-30afe8826b21 service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:57:34.083531 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167631b-d10f-474f-8626-30afe8826b21 service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:57:34.083829 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167631b-d10f-474f-8626-30afe8826b21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:57:34.085591 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-1167631b-d10f-474f-8626-30afe8826b21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:57:34.098219 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167631b-d10f-474f-8626-30afe8826b21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:57:34.099848 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167631b-d10f-474f-8626-30afe8826b21 service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:57:34.111395 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1382/2770] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:57:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:57:37.856123 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15c5c9-5951-4a39-be7f-7bf658cf4997 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:57:37.865308 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb15c5c9-5951-4a39-be7f-7bf658cf4997 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:57:37.868866 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a16914-4473-4ea0-ac8e-50040bd40124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:57:37.869511 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a16914-4473-4ea0-ac8e-50040bd40124 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:57:37.869798 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a16914-4473-4ea0-ac8e-50040bd40124 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:57:37.870077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a16914-4473-4ea0-ac8e-50040bd40124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:57:37.871876 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d5a16914-4473-4ea0-ac8e-50040bd40124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:57:37.883756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a16914-4473-4ea0-ac8e-50040bd40124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:57:37.885259 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a16914-4473-4ea0-ac8e-50040bd40124 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:57:37.895750 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1389/2771] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:57:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:57:41.284719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167631b-d10f-474f-8626-30afe8826b21 service nova] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:57:41.297214 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167631b-d10f-474f-8626-30afe8826b21 service nova] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:57:41.303210 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ef183-5bea-4f23-b02d-aa832ac86566 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:57:41.303861 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ef183-5bea-4f23-b02d-aa832ac86566 service placement] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:57:41.304143 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ef183-5bea-4f23-b02d-aa832ac86566 service placement] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:57:41.304414 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ef183-5bea-4f23-b02d-aa832ac86566 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:57:41.306291 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b11ef183-5bea-4f23-b02d-aa832ac86566 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:57:41.318674 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ef183-5bea-4f23-b02d-aa832ac86566 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:57:41.320316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ef183-5bea-4f23-b02d-aa832ac86566 service placement] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:57:41.331475 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1383/2772] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:57:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:57:44.274428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a16914-4473-4ea0-ac8e-50040bd40124 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:57:44.287341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a16914-4473-4ea0-ac8e-50040bd40124 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:57:44.292851 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26504e-8818-41ea-a21b-026d1e632dbc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:57:44.293510 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26504e-8818-41ea-a21b-026d1e632dbc service placement] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:57:44.293808 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26504e-8818-41ea-a21b-026d1e632dbc service placement] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:57:44.294083 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26504e-8818-41ea-a21b-026d1e632dbc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:57:44.295877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ad26504e-8818-41ea-a21b-026d1e632dbc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:57:44.307930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad26504e-8818-41ea-a21b-026d1e632dbc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:57:44.309497 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad26504e-8818-41ea-a21b-026d1e632dbc service placement] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:57:44.320293 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1390/2773] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:57:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:57:44.338839 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ef183-5bea-4f23-b02d-aa832ac86566 service placement] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:57:44.341079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b11ef183-5bea-4f23-b02d-aa832ac86566 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:57:44.355650 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ef183-5bea-4f23-b02d-aa832ac86566 service placement] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:57:44.362643 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93558474-b1d2-47a2-9cc0-fc161c3f0e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:57:44.363765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93558474-b1d2-47a2-9cc0-fc161c3f0e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:57:44.364051 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93558474-b1d2-47a2-9cc0-fc161c3f0e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:57:44.364347 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93558474-b1d2-47a2-9cc0-fc161c3f0e51 admin admin] PATH_INFO: `/v3/users/b894a36d43224aba9fc4b14b151e75b2` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:57:44.365592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93558474-b1d2-47a2-9cc0-fc161c3f0e51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b894a36d43224aba9fc4b14b151e75b2)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:57:44.367892 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93558474-b1d2-47a2-9cc0-fc161c3f0e51 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:57:44.417359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-93558474-b1d2-47a2-9cc0-fc161c3f0e51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b894a36d43224aba9fc4b14b151e75b2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:44.430104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-93558474-b1d2-47a2-9cc0-fc161c3f0e51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b894a36d43224aba9fc4b14b151e75b2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:44.437606 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-93558474-b1d2-47a2-9cc0-fc161c3f0e51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b894a36d43224aba9fc4b14b151e75b2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:44.438377 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-93558474-b1d2-47a2-9cc0-fc161c3f0e51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b894a36d43224aba9fc4b14b151e75b2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:44.439053 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-93558474-b1d2-47a2-9cc0-fc161c3f0e51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b894a36d43224aba9fc4b14b151e75b2'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:44.453441 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1384/2774] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:57:44 2022] DELETE /identity/v3/users/b894a36d43224aba9fc4b14b151e75b2 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:57:44.637375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26504e-8818-41ea-a21b-026d1e632dbc service placement] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:57:44.653526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26504e-8818-41ea-a21b-026d1e632dbc service placement] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:57:44.712594 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:57:44.713398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:57:44.713676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:57:44.713943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee admin admin] PATH_INFO: `/v3/projects/6c683289fa724e71bb7a9f69154c84a1` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:57:44.715183 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c683289fa724e71bb7a9f69154c84a1)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:57:44.717085 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:57:44.768209 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c683289fa724e71bb7a9f69154c84a1', 'request_id': 'req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:44.773693 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c683289fa724e71bb7a9f69154c84a1', 'request_id': 'req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:44.783311 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c683289fa724e71bb7a9f69154c84a1', 'request_id': 'req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:44.794807 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee admin admin] The token cache is being invalidate because project 6c683289fa724e71bb7a9f69154c84a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:57:44.795120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:44.795771 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:44.798801 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1391/2775] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:57:44 2022] DELETE /identity/v3/projects/6c683289fa724e71bb7a9f69154c84a1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:57:44.803152 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93558474-b1d2-47a2-9cc0-fc161c3f0e51 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:57:44.804919 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-93558474-b1d2-47a2-9cc0-fc161c3f0e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:57:44.818442 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93558474-b1d2-47a2-9cc0-fc161c3f0e51 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:57:44.882332 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978aa87-2442-482c-b656-2c2d037cb487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:57:44.884296 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978aa87-2442-482c-b656-2c2d037cb487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:57:44.884793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978aa87-2442-482c-b656-2c2d037cb487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:57:44.885305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978aa87-2442-482c-b656-2c2d037cb487 admin admin] PATH_INFO: `/v3/users/a9af99a034584fbc8820ad0f863fcc57` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:57:44.886954 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978aa87-2442-482c-b656-2c2d037cb487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9af99a034584fbc8820ad0f863fcc57)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:57:44.889228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978aa87-2442-482c-b656-2c2d037cb487 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:57:44.941656 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1978aa87-2442-482c-b656-2c2d037cb487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9af99a034584fbc8820ad0f863fcc57'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:44.942762 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:57:44.956462 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1978aa87-2442-482c-b656-2c2d037cb487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9af99a034584fbc8820ad0f863fcc57'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:44.963949 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1978aa87-2442-482c-b656-2c2d037cb487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9af99a034584fbc8820ad0f863fcc57'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:44.964706 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1978aa87-2442-482c-b656-2c2d037cb487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9af99a034584fbc8820ad0f863fcc57'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:44.965352 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-1978aa87-2442-482c-b656-2c2d037cb487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9af99a034584fbc8820ad0f863fcc57'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:44.980348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1385/2776] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:57:44 2022] DELETE /identity/v3/users/a9af99a034584fbc8820ad0f863fcc57 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:57:45.159549 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:57:45.161380 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:57:45.170035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:57:45.177994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2adccd-a9d8-43d7-95ec-3d2b05c794ee admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:57:45.242017 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb721fb9-7775-4ba5-80f0-ce69394d4089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:57:45.242918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb721fb9-7775-4ba5-80f0-ce69394d4089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:57:45.243229 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb721fb9-7775-4ba5-80f0-ce69394d4089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:57:45.243502 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb721fb9-7775-4ba5-80f0-ce69394d4089 admin admin] PATH_INFO: `/v3/projects/a0c485077baa456cb7f8ce777f516fa9` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:57:45.244769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb721fb9-7775-4ba5-80f0-ce69394d4089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0c485077baa456cb7f8ce777f516fa9)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:57:45.246722 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb721fb9-7775-4ba5-80f0-ce69394d4089 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:57:45.299376 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-bb721fb9-7775-4ba5-80f0-ce69394d4089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0c485077baa456cb7f8ce777f516fa9', 'request_id': 'req-bb721fb9-7775-4ba5-80f0-ce69394d4089'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:45.305243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-bb721fb9-7775-4ba5-80f0-ce69394d4089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0c485077baa456cb7f8ce777f516fa9', 'request_id': 'req-bb721fb9-7775-4ba5-80f0-ce69394d4089'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:45.314878 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-bb721fb9-7775-4ba5-80f0-ce69394d4089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0c485077baa456cb7f8ce777f516fa9', 'request_id': 'req-bb721fb9-7775-4ba5-80f0-ce69394d4089'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:45.330607 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-bb721fb9-7775-4ba5-80f0-ce69394d4089 admin admin] The token cache is being invalidate because project a0c485077baa456cb7f8ce777f516fa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:57:45.330907 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-bb721fb9-7775-4ba5-80f0-ce69394d4089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:45.331778 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-bb721fb9-7775-4ba5-80f0-ce69394d4089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:57:45.333468 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1392/2777] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:57:45 2022] DELETE /identity/v3/projects/a0c485077baa456cb7f8ce777f516fa9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:57:54.977261 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=12,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:57:55.326757 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:58:10.750752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978aa87-2442-482c-b656-2c2d037cb487 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:58:10.752381 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-1978aa87-2442-482c-b656-2c2d037cb487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:58:10.812246 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978aa87-2442-482c-b656-2c2d037cb487 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:58:10.862416 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b267925-573d-41af-9dd1-71b4ec324de4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:58:10.863137 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b267925-573d-41af-9dd1-71b4ec324de4 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:58:10.863421 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b267925-573d-41af-9dd1-71b4ec324de4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:58:10.863696 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b267925-573d-41af-9dd1-71b4ec324de4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:58:10.865610 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-6b267925-573d-41af-9dd1-71b4ec324de4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:58:10.931256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b267925-573d-41af-9dd1-71b4ec324de4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:58:10.932705 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b267925-573d-41af-9dd1-71b4ec324de4 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:58:10.942494 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1386/2778] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:58:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:58:17.345639 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb721fb9-7775-4ba5-80f0-ce69394d4089 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:58:17.347148 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-bb721fb9-7775-4ba5-80f0-ce69394d4089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:58:17.360257 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb721fb9-7775-4ba5-80f0-ce69394d4089 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:58:17.365159 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b0c45-bf45-4dc8-870b-b06e08df2f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:58:17.366037 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b0c45-bf45-4dc8-870b-b06e08df2f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:58:17.366323 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b0c45-bf45-4dc8-870b-b06e08df2f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:58:17.366657 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b0c45-bf45-4dc8-870b-b06e08df2f8e admin admin] PATH_INFO: `/v3/users/17108265cfba44b29c362ac18bbf3e20` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:58:17.368010 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b0c45-bf45-4dc8-870b-b06e08df2f8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=17108265cfba44b29c362ac18bbf3e20)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:58:17.369971 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b0c45-bf45-4dc8-870b-b06e08df2f8e admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:58:17.420548 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-213b0c45-bf45-4dc8-870b-b06e08df2f8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17108265cfba44b29c362ac18bbf3e20'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:17.437542 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-213b0c45-bf45-4dc8-870b-b06e08df2f8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17108265cfba44b29c362ac18bbf3e20'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:17.444616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-213b0c45-bf45-4dc8-870b-b06e08df2f8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17108265cfba44b29c362ac18bbf3e20'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:17.445513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-213b0c45-bf45-4dc8-870b-b06e08df2f8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17108265cfba44b29c362ac18bbf3e20'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:17.446221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-213b0c45-bf45-4dc8-870b-b06e08df2f8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17108265cfba44b29c362ac18bbf3e20'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:17.465704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1393/2779] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:58:17 2022] DELETE /identity/v3/users/17108265cfba44b29c362ac18bbf3e20 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:58:17.648572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b267925-573d-41af-9dd1-71b4ec324de4 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:58:17.660046 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b267925-573d-41af-9dd1-71b4ec324de4 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:58:17.719240 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:58:17.720118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:58:17.720401 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:58:17.720670 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac admin admin] PATH_INFO: `/v3/projects/0c6f9207778b4e0fbdbbae28c8f7b224` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:58:17.721879 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c6f9207778b4e0fbdbbae28c8f7b224)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:58:17.723851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:58:17.731858 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:58:17.771872 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c6f9207778b4e0fbdbbae28c8f7b224', 'request_id': 'req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:17.778056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c6f9207778b4e0fbdbbae28c8f7b224', 'request_id': 'req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:17.784646 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c6f9207778b4e0fbdbbae28c8f7b224', 'request_id': 'req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:17.799420 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac admin admin] The token cache is being invalidate because project 0c6f9207778b4e0fbdbbae28c8f7b224 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:58:17.799704 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:17.800319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:17.801878 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1387/2780] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:58:17 2022] DELETE /identity/v3/projects/0c6f9207778b4e0fbdbbae28c8f7b224 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:58:17.806009 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b0c45-bf45-4dc8-870b-b06e08df2f8e admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:58:17.807516 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-213b0c45-bf45-4dc8-870b-b06e08df2f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:58:17.821218 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b0c45-bf45-4dc8-870b-b06e08df2f8e admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:58:17.881203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614af00-52ed-491f-87af-786fae20ed61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:58:17.882079 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614af00-52ed-491f-87af-786fae20ed61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:58:17.882352 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614af00-52ed-491f-87af-786fae20ed61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:58:17.882618 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614af00-52ed-491f-87af-786fae20ed61 admin admin] PATH_INFO: `/v3/users/5d08b387d141426eb831137934589a2e` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:58:17.883964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614af00-52ed-491f-87af-786fae20ed61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d08b387d141426eb831137934589a2e)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:58:17.885889 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614af00-52ed-491f-87af-786fae20ed61 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:58:17.934404 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3614af00-52ed-491f-87af-786fae20ed61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d08b387d141426eb831137934589a2e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:17.946301 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3614af00-52ed-491f-87af-786fae20ed61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d08b387d141426eb831137934589a2e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:17.953573 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3614af00-52ed-491f-87af-786fae20ed61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d08b387d141426eb831137934589a2e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:17.954398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3614af00-52ed-491f-87af-786fae20ed61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d08b387d141426eb831137934589a2e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:17.955143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-3614af00-52ed-491f-87af-786fae20ed61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d08b387d141426eb831137934589a2e'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:17.956843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:58:17.971576 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1394/2781] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 05:58:17 2022] DELETE /identity/v3/users/5d08b387d141426eb831137934589a2e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:58:18.149692 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:58:18.151578 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:58:18.164878 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51a6ef-a39b-4cee-b0d0-51c6dbce40ac admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:58:18.223587 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e693fba-1b57-43e3-9f9a-743b8b03bbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:58:18.224478 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e693fba-1b57-43e3-9f9a-743b8b03bbe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:58:18.224764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e693fba-1b57-43e3-9f9a-743b8b03bbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:58:18.225042 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e693fba-1b57-43e3-9f9a-743b8b03bbe3 admin admin] PATH_INFO: `/v3/projects/de984aea424f468e8eb9d92d44ccbad2` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:58:18.226232 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e693fba-1b57-43e3-9f9a-743b8b03bbe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de984aea424f468e8eb9d92d44ccbad2)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:58:18.228242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e693fba-1b57-43e3-9f9a-743b8b03bbe3 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:58:18.276203 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3e693fba-1b57-43e3-9f9a-743b8b03bbe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de984aea424f468e8eb9d92d44ccbad2', 'request_id': 'req-3e693fba-1b57-43e3-9f9a-743b8b03bbe3'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:18.282603 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3e693fba-1b57-43e3-9f9a-743b8b03bbe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de984aea424f468e8eb9d92d44ccbad2', 'request_id': 'req-3e693fba-1b57-43e3-9f9a-743b8b03bbe3'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:18.287972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3e693fba-1b57-43e3-9f9a-743b8b03bbe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de984aea424f468e8eb9d92d44ccbad2', 'request_id': 'req-3e693fba-1b57-43e3-9f9a-743b8b03bbe3'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:18.305069 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3e693fba-1b57-43e3-9f9a-743b8b03bbe3 admin admin] The token cache is being invalidate because project de984aea424f468e8eb9d92d44ccbad2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 05:58:18.305359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3e693fba-1b57-43e3-9f9a-743b8b03bbe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:18.306052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3e693fba-1b57-43e3-9f9a-743b8b03bbe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 05:58:18.307725 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1388/2782] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 05:58:18 2022] DELETE /identity/v3/projects/de984aea424f468e8eb9d92d44ccbad2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 05:58:27.972711 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:58:28.300388 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=72,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 05:58:40.907192 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614af00-52ed-491f-87af-786fae20ed61 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 05:58:40.909036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-3614af00-52ed-491f-87af-786fae20ed61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:58:40.967574 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614af00-52ed-491f-87af-786fae20ed61 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 05:58:41.019667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9f908-f3ae-406e-8463-345f5628666d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 05:58:41.020309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9f908-f3ae-406e-8463-345f5628666d service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 05:58:41.020587 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9f908-f3ae-406e-8463-345f5628666d service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 05:58:41.020860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9f908-f3ae-406e-8463-345f5628666d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 05:58:41.022842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-7fa9f908-f3ae-406e-8463-345f5628666d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 05:58:41.096036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9f908-f3ae-406e-8463-345f5628666d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 05:58:41.097535 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9f908-f3ae-406e-8463-345f5628666d service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 05:58:41.109727 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1395/2783] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 05:58:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 05:58:51.104406 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=74 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:00:01.184438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021748a-6dee-4d34-a588-dbd02913973c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:00:01.185002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021748a-6dee-4d34-a588-dbd02913973c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:00:01.185002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021748a-6dee-4d34-a588-dbd02913973c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:00:01.203394 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-5021748a-6dee-4d34-a588-dbd02913973c None None] MFA Rules not processed for user `e5a03f7524f7436fbbd78a7778702d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:00:01.204643 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5021748a-6dee-4d34-a588-dbd02913973c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:00:01.261481 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1389/2784] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:00:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:00:01.573686 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9f908-f3ae-406e-8463-345f5628666d service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:00:01.575389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-7fa9f908-f3ae-406e-8463-345f5628666d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:00:01.587337 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9f908-f3ae-406e-8463-345f5628666d service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:00:01.591478 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276cc42-e78a-4d6c-8584-baa0e9256135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:00:01.592126 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276cc42-e78a-4d6c-8584-baa0e9256135 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:00:01.592419 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276cc42-e78a-4d6c-8584-baa0e9256135 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:00:01.592692 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276cc42-e78a-4d6c-8584-baa0e9256135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:00:01.594790 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4276cc42-e78a-4d6c-8584-baa0e9256135 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:00:01.605117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4276cc42-e78a-4d6c-8584-baa0e9256135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:00:01.606557 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4276cc42-e78a-4d6c-8584-baa0e9256135 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:00:01.614725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1396/2785] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:00:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:00:07.995162 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021748a-6dee-4d34-a588-dbd02913973c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:00:08.006322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021748a-6dee-4d34-a588-dbd02913973c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:00:08.009937 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a47383-e4c2-4976-beb2-501b5805b0e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:00:08.010629 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a47383-e4c2-4976-beb2-501b5805b0e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:00:08.010911 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a47383-e4c2-4976-beb2-501b5805b0e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:00:08.011202 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a47383-e4c2-4976-beb2-501b5805b0e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:00:08.013934 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-73a47383-e4c2-4976-beb2-501b5805b0e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:00:08.025196 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a47383-e4c2-4976-beb2-501b5805b0e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:00:08.026940 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a47383-e4c2-4976-beb2-501b5805b0e6 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:00:08.036915 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1390/2786] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:00:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:00:11.611253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=6 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:00:18.035828 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=31 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:01:22.701590 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276cc42-e78a-4d6c-8584-baa0e9256135 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:01:22.703253 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4276cc42-e78a-4d6c-8584-baa0e9256135 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:01:22.719828 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276cc42-e78a-4d6c-8584-baa0e9256135 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:01:22.725859 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d325f-3a63-44f5-b6c7-c440cdaa1142 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:01:22.726500 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d325f-3a63-44f5-b6c7-c440cdaa1142 service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:01:22.726788 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d325f-3a63-44f5-b6c7-c440cdaa1142 service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:01:22.727061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d325f-3a63-44f5-b6c7-c440cdaa1142 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:01:22.729027 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-108d325f-3a63-44f5-b6c7-c440cdaa1142 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:01:22.740079 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d325f-3a63-44f5-b6c7-c440cdaa1142 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:01:22.741662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d325f-3a63-44f5-b6c7-c440cdaa1142 service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:01:22.751071 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1397/2787] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:01:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:01:32.746812 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=6 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:01:47.616645 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a47383-e4c2-4976-beb2-501b5805b0e6 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:01:47.679699 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a47383-e4c2-4976-beb2-501b5805b0e6 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:01:47.686599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0466d-88ec-4ebc-9a1b-1e463a4cf7a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:01:47.687629 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0466d-88ec-4ebc-9a1b-1e463a4cf7a0 service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:01:47.688197 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0466d-88ec-4ebc-9a1b-1e463a4cf7a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:01:47.688595 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0466d-88ec-4ebc-9a1b-1e463a4cf7a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:01:47.691243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3db0466d-88ec-4ebc-9a1b-1e463a4cf7a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:01:47.735771 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db0466d-88ec-4ebc-9a1b-1e463a4cf7a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:01:47.737605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db0466d-88ec-4ebc-9a1b-1e463a4cf7a0 service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:01:47.754201 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1391/2788] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:01:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:01:52.484784 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d325f-3a63-44f5-b6c7-c440cdaa1142 service nova] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:01:52.506114 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d325f-3a63-44f5-b6c7-c440cdaa1142 service nova] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:01:52.510222 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4581eb-475e-4ac8-8122-97dc8bfbed21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:01:52.510866 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4581eb-475e-4ac8-8122-97dc8bfbed21 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:01:52.511209 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4581eb-475e-4ac8-8122-97dc8bfbed21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:01:52.511486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4581eb-475e-4ac8-8122-97dc8bfbed21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:01:52.513389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6c4581eb-475e-4ac8-8122-97dc8bfbed21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:01:52.606663 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4581eb-475e-4ac8-8122-97dc8bfbed21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:01:52.608168 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4581eb-475e-4ac8-8122-97dc8bfbed21 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:01:52.628237 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1398/2789] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:01:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:01:55.825364 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0466d-88ec-4ebc-9a1b-1e463a4cf7a0 service nova] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:01:55.827144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-3db0466d-88ec-4ebc-9a1b-1e463a4cf7a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:01:55.841722 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0466d-88ec-4ebc-9a1b-1e463a4cf7a0 service nova] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:01:55.847552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde71c3-c0ed-494c-8502-c0a7d66e4a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:01:55.848431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde71c3-c0ed-494c-8502-c0a7d66e4a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:01:55.848704 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde71c3-c0ed-494c-8502-c0a7d66e4a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:01:55.848975 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde71c3-c0ed-494c-8502-c0a7d66e4a9c admin admin] PATH_INFO: `/v3/users/e5a03f7524f7436fbbd78a7778702d1b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:01:55.850275 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde71c3-c0ed-494c-8502-c0a7d66e4a9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5a03f7524f7436fbbd78a7778702d1b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:01:55.852307 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde71c3-c0ed-494c-8502-c0a7d66e4a9c admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:01:55.900495 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3fde71c3-c0ed-494c-8502-c0a7d66e4a9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5a03f7524f7436fbbd78a7778702d1b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:01:55.912271 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3fde71c3-c0ed-494c-8502-c0a7d66e4a9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5a03f7524f7436fbbd78a7778702d1b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:01:55.918951 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3fde71c3-c0ed-494c-8502-c0a7d66e4a9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5a03f7524f7436fbbd78a7778702d1b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:01:55.919613 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3fde71c3-c0ed-494c-8502-c0a7d66e4a9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5a03f7524f7436fbbd78a7778702d1b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:01:55.920215 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-3fde71c3-c0ed-494c-8502-c0a7d66e4a9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5a03f7524f7436fbbd78a7778702d1b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:01:55.934243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1392/2790] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 06:01:55 2022] DELETE /identity/v3/users/e5a03f7524f7436fbbd78a7778702d1b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:01:56.110382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4581eb-475e-4ac8-8122-97dc8bfbed21 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:01:56.120401 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4581eb-475e-4ac8-8122-97dc8bfbed21 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:01:56.180632 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65714bf9-4650-4138-8138-909017f65c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:01:56.181675 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65714bf9-4650-4138-8138-909017f65c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:01:56.181975 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65714bf9-4650-4138-8138-909017f65c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:01:56.182217 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65714bf9-4650-4138-8138-909017f65c43 admin admin] PATH_INFO: `/v3/projects/a3f30d9c29794ccfb52188216cd48085` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:01:56.183740 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65714bf9-4650-4138-8138-909017f65c43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3f30d9c29794ccfb52188216cd48085)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:01:56.185707 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65714bf9-4650-4138-8138-909017f65c43 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:01:56.237930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-65714bf9-4650-4138-8138-909017f65c43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3f30d9c29794ccfb52188216cd48085', 'request_id': 'req-65714bf9-4650-4138-8138-909017f65c43'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:01:56.242417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:01:56.243851 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-65714bf9-4650-4138-8138-909017f65c43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3f30d9c29794ccfb52188216cd48085', 'request_id': 'req-65714bf9-4650-4138-8138-909017f65c43'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:01:56.254597 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-65714bf9-4650-4138-8138-909017f65c43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3f30d9c29794ccfb52188216cd48085', 'request_id': 'req-65714bf9-4650-4138-8138-909017f65c43'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:01:56.266904 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-65714bf9-4650-4138-8138-909017f65c43 admin admin] The token cache is being invalidate because project a3f30d9c29794ccfb52188216cd48085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:01:56.267268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-65714bf9-4650-4138-8138-909017f65c43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:01:56.267980 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-65714bf9-4650-4138-8138-909017f65c43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:01:56.271093 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1399/2791] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 06:01:56 2022] DELETE /identity/v3/projects/a3f30d9c29794ccfb52188216cd48085 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:01:56.274946 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daf5fe-c749-47d2-b5ff-dac6a27833b0 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:01:56.275425 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daf5fe-c749-47d2-b5ff-dac6a27833b0 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:01:56.275791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daf5fe-c749-47d2-b5ff-dac6a27833b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:01:56.334516 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-b7daf5fe-c749-47d2-b5ff-dac6a27833b0 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:01:56.335809 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b7daf5fe-c749-47d2-b5ff-dac6a27833b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:01:56.368660 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:01:56.397024 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1393/2792] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 21 06:01:56.404826 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c2743-9819-41bf-bcb5-5f65d5dfbcbf None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:01:56.405134 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c2743-9819-41bf-bcb5-5f65d5dfbcbf None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:01:56.405408 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c2743-9819-41bf-bcb5-5f65d5dfbcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:01:56.423704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-cc5c2743-9819-41bf-bcb5-5f65d5dfbcbf None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:01:56.424899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-cc5c2743-9819-41bf-bcb5-5f65d5dfbcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:01:56.432791 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1400/2793] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:01:56.438400 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7daf5fe-c749-47d2-b5ff-dac6a27833b0 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:01:56.448346 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7daf5fe-c749-47d2-b5ff-dac6a27833b0 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:01:56.453291 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b135fca-49c7-428e-8b0f-af3da82684f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:01:56.453943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b135fca-49c7-428e-8b0f-af3da82684f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:01:56.454226 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b135fca-49c7-428e-8b0f-af3da82684f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:01:56.454498 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b135fca-49c7-428e-8b0f-af3da82684f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:01:56.455332 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b135fca-49c7-428e-8b0f-af3da82684f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:01:56.455835 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b135fca-49c7-428e-8b0f-af3da82684f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:01:56.458735 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b135fca-49c7-428e-8b0f-af3da82684f0 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:01:56.473803 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1394/2794] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 06:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:01:56.478180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c2743-9819-41bf-bcb5-5f65d5dfbcbf None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:01:56.488193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c2743-9819-41bf-bcb5-5f65d5dfbcbf None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:01:56.494689 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47222c34-75b0-4b98-9af3-dc3c047a94a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:01:56.495395 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47222c34-75b0-4b98-9af3-dc3c047a94a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:01:56.495671 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47222c34-75b0-4b98-9af3-dc3c047a94a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:01:56.495916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47222c34-75b0-4b98-9af3-dc3c047a94a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:01:56.496627 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47222c34-75b0-4b98-9af3-dc3c047a94a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:01:56.498530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47222c34-75b0-4b98-9af3-dc3c047a94a8 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:01:56.526025 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1401/2795] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 06:01:56 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:01:56.530395 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b135fca-49c7-428e-8b0f-af3da82684f0 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:01:56.541516 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b135fca-49c7-428e-8b0f-af3da82684f0 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:01:56.596472 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbf7b0-7db6-4096-85f8-863c52c6c51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:01:56.597124 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbf7b0-7db6-4096-85f8-863c52c6c51d admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:01:56.597417 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbf7b0-7db6-4096-85f8-863c52c6c51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:01:56.597688 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbf7b0-7db6-4096-85f8-863c52c6c51d admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:01:56.598373 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbf7b0-7db6-4096-85f8-863c52c6c51d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:01:56.600340 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbf7b0-7db6-4096-85f8-863c52c6c51d admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:01:56.622030 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1395/2796] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 06:01:56 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:01:56.626342 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47222c34-75b0-4b98-9af3-dc3c047a94a8 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:01:56.637517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47222c34-75b0-4b98-9af3-dc3c047a94a8 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:01:56.642616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3de02-23b2-43a5-b84d-1ecc3c405525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:01:56.643310 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3de02-23b2-43a5-b84d-1ecc3c405525 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:01:56.643622 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3de02-23b2-43a5-b84d-1ecc3c405525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:01:56.643897 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3de02-23b2-43a5-b84d-1ecc3c405525 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:01:56.644629 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3de02-23b2-43a5-b84d-1ecc3c405525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:01:56.646241 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae3de02-23b2-43a5-b84d-1ecc3c405525 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:01:56.658106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1402/2797] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 06:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:01:56.662079 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbf7b0-7db6-4096-85f8-863c52c6c51d admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:01:56.672984 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbf7b0-7db6-4096-85f8-863c52c6c51d admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:01:56.678666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd4228-ac60-463a-a8d0-dfc923d1d2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:01:56.679372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd4228-ac60-463a-a8d0-dfc923d1d2f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:01:56.679658 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd4228-ac60-463a-a8d0-dfc923d1d2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:01:56.679933 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd4228-ac60-463a-a8d0-dfc923d1d2f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:01:56.680666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd4228-ac60-463a-a8d0-dfc923d1d2f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:01:56.682294 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd4228-ac60-463a-a8d0-dfc923d1d2f6 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:01:56.693195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1396/2798] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 06:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:01:56.697341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3de02-23b2-43a5-b84d-1ecc3c405525 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:01:56.707586 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3de02-23b2-43a5-b84d-1ecc3c405525 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:01:56.714856 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52323da-4b9f-4fc2-aa6e-c7f2cf986782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:01:56.716268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52323da-4b9f-4fc2-aa6e-c7f2cf986782 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:01:56.716599 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52323da-4b9f-4fc2-aa6e-c7f2cf986782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:01:56.716946 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52323da-4b9f-4fc2-aa6e-c7f2cf986782 admin admin] PATH_INFO: `/v3/projects/1c87efe1b2b74d7d8749ae5e14bdd412/users/631157435dd242be80eb9f9fbf89ae76/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:01:56.733665 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52323da-4b9f-4fc2-aa6e-c7f2cf986782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c87efe1b2b74d7d8749ae5e14bdd412, user_id=631157435dd242be80eb9f9fbf89ae76, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:01:56.736429 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52323da-4b9f-4fc2-aa6e-c7f2cf986782 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:01:56.751676 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1403/2799] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 06:01:56 2022] PUT /identity/v3/projects/1c87efe1b2b74d7d8749ae5e14bdd412/users/631157435dd242be80eb9f9fbf89ae76/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:01:56.756860 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4edfc-9df2-437d-bc82-2665bedcad9f None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:01:56.757185 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4edfc-9df2-437d-bc82-2665bedcad9f None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:01:56.757488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4edfc-9df2-437d-bc82-2665bedcad9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:01:56.776144 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-5ee4edfc-9df2-437d-bc82-2665bedcad9f None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:01:56.777654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-5ee4edfc-9df2-437d-bc82-2665bedcad9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:01:56.838391 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1397/2800] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:01:56.848298 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52323da-4b9f-4fc2-aa6e-c7f2cf986782 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:01:56.850128 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c52323da-4b9f-4fc2-aa6e-c7f2cf986782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:01:56.913583 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52323da-4b9f-4fc2-aa6e-c7f2cf986782 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:01:56.918163 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbde2c-e086-4e16-9b6f-31eba3678083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:01:56.918809 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbde2c-e086-4e16-9b6f-31eba3678083 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:01:56.919382 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbde2c-e086-4e16-9b6f-31eba3678083 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:01:56.919666 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbde2c-e086-4e16-9b6f-31eba3678083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:01:56.929390 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cbde2c-e086-4e16-9b6f-31eba3678083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:01:56.931141 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cbde2c-e086-4e16-9b6f-31eba3678083 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:01:56.940903 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1404/2801] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:01:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:02:00.914586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817064b-60d6-4766-bf9d-eb3886964a2c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:02:00.915129 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817064b-60d6-4766-bf9d-eb3886964a2c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:02:00.915245 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817064b-60d6-4766-bf9d-eb3886964a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:02:00.945184 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-1817064b-60d6-4766-bf9d-eb3886964a2c None None] MFA Rules not processed for user `631157435dd242be80eb9f9fbf89ae76`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:02:00.946409 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-1817064b-60d6-4766-bf9d-eb3886964a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:02:00.983658 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:02:01.028712 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1398/2802] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:02:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:02:01.045911 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbde2c-e086-4e16-9b6f-31eba3678083 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:02:01.048813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-77cbde2c-e086-4e16-9b6f-31eba3678083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:02:01.151496 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:02:01.169381 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbde2c-e086-4e16-9b6f-31eba3678083 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:02:01.175390 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dad4b8-e03c-4ffe-ba1e-780feea39868 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:02:01.176061 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dad4b8-e03c-4ffe-ba1e-780feea39868 service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:02:01.176348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dad4b8-e03c-4ffe-ba1e-780feea39868 service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:02:01.176621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dad4b8-e03c-4ffe-ba1e-780feea39868 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:02:01.185428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dad4b8-e03c-4ffe-ba1e-780feea39868 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:02:01.187107 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dad4b8-e03c-4ffe-ba1e-780feea39868 service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:02:01.196997 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1405/2803] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:02:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:02:02.522933 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011034a-e98d-4719-87aa-7d19e41cfdfe None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:02:02.523484 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011034a-e98d-4719-87aa-7d19e41cfdfe None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:02:02.523586 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011034a-e98d-4719-87aa-7d19e41cfdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:02:02.541967 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-a011034a-e98d-4719-87aa-7d19e41cfdfe None None] MFA Rules not processed for user `631157435dd242be80eb9f9fbf89ae76`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:02:02.543216 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a011034a-e98d-4719-87aa-7d19e41cfdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:02:02.550230 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1399/2804] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:02:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:02:03.158899 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dad4b8-e03c-4ffe-ba1e-780feea39868 service nova] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:02:03.160535 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-d8dad4b8-e03c-4ffe-ba1e-780feea39868 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:02:03.264018 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dad4b8-e03c-4ffe-ba1e-780feea39868 service nova] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:02:03.269044 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a064288-d4fe-41f4-ac48-6c687183eb4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:02:03.269704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a064288-d4fe-41f4-ac48-6c687183eb4a service placement] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:02:03.269985 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a064288-d4fe-41f4-ac48-6c687183eb4a service placement] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:02:03.270260 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a064288-d4fe-41f4-ac48-6c687183eb4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:02:03.272072 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6a064288-d4fe-41f4-ac48-6c687183eb4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:02:03.284255 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a064288-d4fe-41f4-ac48-6c687183eb4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:02:03.285853 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a064288-d4fe-41f4-ac48-6c687183eb4a service placement] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:02:03.296203 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1406/2805] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:02:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:02:04.223309 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011034a-e98d-4719-87aa-7d19e41cfdfe None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:02:04.224945 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a011034a-e98d-4719-87aa-7d19e41cfdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:02:04.241552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011034a-e98d-4719-87aa-7d19e41cfdfe None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:02:04.247649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160625e-8817-4fc7-81c8-3cf1e4bbd08d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:02:04.248316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160625e-8817-4fc7-81c8-3cf1e4bbd08d service placement] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:02:04.248723 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160625e-8817-4fc7-81c8-3cf1e4bbd08d service placement] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:02:04.248857 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160625e-8817-4fc7-81c8-3cf1e4bbd08d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:02:04.250793 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4160625e-8817-4fc7-81c8-3cf1e4bbd08d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:02:04.263723 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160625e-8817-4fc7-81c8-3cf1e4bbd08d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:02:04.266654 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160625e-8817-4fc7-81c8-3cf1e4bbd08d service placement] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:02:04.285505 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1400/2806] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:02:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:02:13.291715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=49 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:02:14.275156 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=20 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:02:15.377334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a064288-d4fe-41f4-ac48-6c687183eb4a service placement] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:02:15.392786 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a064288-d4fe-41f4-ac48-6c687183eb4a service placement] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:02:15.399486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a0d4b-fade-45e8-8b70-d79cdcf2e45c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:02:15.400146 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a0d4b-fade-45e8-8b70-d79cdcf2e45c service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:02:15.400432 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a0d4b-fade-45e8-8b70-d79cdcf2e45c service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:02:15.400704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a0d4b-fade-45e8-8b70-d79cdcf2e45c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:02:15.402600 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-ce5a0d4b-fade-45e8-8b70-d79cdcf2e45c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:02:15.416377 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a0d4b-fade-45e8-8b70-d79cdcf2e45c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:02:15.417979 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a0d4b-fade-45e8-8b70-d79cdcf2e45c service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:02:15.429261 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1407/2807] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:02:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:02:25.424651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=6 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:03:34.917450 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46582e6a-f31e-4b23-87a9-17f5cd6a0808 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:03:34.918020 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46582e6a-f31e-4b23-87a9-17f5cd6a0808 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:03:34.918020 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46582e6a-f31e-4b23-87a9-17f5cd6a0808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:03:34.935801 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-46582e6a-f31e-4b23-87a9-17f5cd6a0808 None None] MFA Rules not processed for user `631157435dd242be80eb9f9fbf89ae76`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:03:34.937018 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-46582e6a-f31e-4b23-87a9-17f5cd6a0808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:03:34.944698 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1401/2808] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:03:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:03:40.266647 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a0d4b-fade-45e8-8b70-d79cdcf2e45c service nova] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:03:40.293896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a0d4b-fade-45e8-8b70-d79cdcf2e45c service nova] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:03:40.297860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b79da-c533-4345-8417-76a6a4f4b041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:03:40.298499 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b79da-c533-4345-8417-76a6a4f4b041 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:03:40.298777 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b79da-c533-4345-8417-76a6a4f4b041 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:03:40.299059 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b79da-c533-4345-8417-76a6a4f4b041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:03:40.300987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-730b79da-c533-4345-8417-76a6a4f4b041 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:03:40.314644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b79da-c533-4345-8417-76a6a4f4b041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:03:40.316111 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b79da-c533-4345-8417-76a6a4f4b041 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:03:40.329457 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1408/2809] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:03:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:03:44.930986 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:03:50.322824 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=10 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:05:17.967851 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71433312-3ddc-4955-b670-a1510f84c2ee None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:05:17.968391 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71433312-3ddc-4955-b670-a1510f84c2ee None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:05:17.968391 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71433312-3ddc-4955-b670-a1510f84c2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:05:17.987615 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-71433312-3ddc-4955-b670-a1510f84c2ee None None] MFA Rules not processed for user `631157435dd242be80eb9f9fbf89ae76`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:05:17.988825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-71433312-3ddc-4955-b670-a1510f84c2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:05:17.996194 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1402/2810] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:05:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:05:18.312205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b79da-c533-4345-8417-76a6a4f4b041 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:05:18.323239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b79da-c533-4345-8417-76a6a4f4b041 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:05:18.327817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709c288-46dc-4497-825b-085a9670740c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:05:18.328461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709c288-46dc-4497-825b-085a9670740c service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:05:18.328744 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709c288-46dc-4497-825b-085a9670740c service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:05:18.329015 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709c288-46dc-4497-825b-085a9670740c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:05:18.331179 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-1709c288-46dc-4497-825b-085a9670740c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:05:18.354604 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709c288-46dc-4497-825b-085a9670740c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:05:18.356075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709c288-46dc-4497-825b-085a9670740c service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:05:18.364284 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1409/2811] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:05:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:05:20.508036 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71433312-3ddc-4955-b670-a1510f84c2ee None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:05:20.518975 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71433312-3ddc-4955-b670-a1510f84c2ee None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:05:20.523272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294fd5c-1e47-4212-9605-1f904cf39140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:05:20.523948 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294fd5c-1e47-4212-9605-1f904cf39140 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:05:20.524229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294fd5c-1e47-4212-9605-1f904cf39140 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:05:20.524505 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294fd5c-1e47-4212-9605-1f904cf39140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:05:20.526632 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a294fd5c-1e47-4212-9605-1f904cf39140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:05:20.537699 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294fd5c-1e47-4212-9605-1f904cf39140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:05:20.539195 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294fd5c-1e47-4212-9605-1f904cf39140 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:05:20.548056 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1403/2812] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:05:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:05:28.360585 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=11 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:05:30.544191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=9 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:06:52.142919 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709c288-46dc-4497-825b-085a9670740c service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:06:52.158937 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709c288-46dc-4497-825b-085a9670740c service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:06:52.165282 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2c105-f107-4c83-a48d-923fce79066a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:06:52.166047 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2c105-f107-4c83-a48d-923fce79066a service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:06:52.166378 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2c105-f107-4c83-a48d-923fce79066a service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:06:52.166806 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2c105-f107-4c83-a48d-923fce79066a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:06:52.168963 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-2bd2c105-f107-4c83-a48d-923fce79066a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:06:52.184824 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2c105-f107-4c83-a48d-923fce79066a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:06:52.186486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2c105-f107-4c83-a48d-923fce79066a service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:06:52.198005 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1410/2813] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:06:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:06:55.213693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735959b0-a1cb-4bda-ad6e-055666365a4c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:06:55.214268 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735959b0-a1cb-4bda-ad6e-055666365a4c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:06:55.214268 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735959b0-a1cb-4bda-ad6e-055666365a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:06:55.233018 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-735959b0-a1cb-4bda-ad6e-055666365a4c None None] MFA Rules not processed for user `631157435dd242be80eb9f9fbf89ae76`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:06:55.234224 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-735959b0-a1cb-4bda-ad6e-055666365a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:06:55.241528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1404/2814] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:06:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:07:01.696082 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2c105-f107-4c83-a48d-923fce79066a service nova] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:07:01.708125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2c105-f107-4c83-a48d-923fce79066a service nova] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:07:01.715279 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d90db8-2975-4e24-96db-7ed270989336 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:07:01.715919 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d90db8-2975-4e24-96db-7ed270989336 service nova] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:07:01.716211 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d90db8-2975-4e24-96db-7ed270989336 service nova] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:07:01.716471 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d90db8-2975-4e24-96db-7ed270989336 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:07:01.724678 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d90db8-2975-4e24-96db-7ed270989336 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:07:01.726265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d90db8-2975-4e24-96db-7ed270989336 service nova] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:07:01.735310 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1411/2815] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:07:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:07:05.227604 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:07:11.731411 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=12 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:07:13.161129 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735959b0-a1cb-4bda-ad6e-055666365a4c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:07:13.174726 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735959b0-a1cb-4bda-ad6e-055666365a4c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:07:13.181464 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29365a35-d0ab-4509-ab3f-8a178996886a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:07:13.182113 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29365a35-d0ab-4509-ab3f-8a178996886a service placement] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:07:13.182388 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29365a35-d0ab-4509-ab3f-8a178996886a service placement] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:07:13.182663 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29365a35-d0ab-4509-ab3f-8a178996886a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:07:13.191651 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29365a35-d0ab-4509-ab3f-8a178996886a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:07:13.193242 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29365a35-d0ab-4509-ab3f-8a178996886a service placement] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:07:13.203865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1405/2816] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:07:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:07:23.199358 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=6 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:08:31.303178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d90db8-2975-4e24-96db-7ed270989336 service nova] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:31.313122 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d90db8-2975-4e24-96db-7ed270989336 service nova] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:31.317694 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578b83d-835a-47c9-9d38-b8b11f4908a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:31.318330 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578b83d-835a-47c9-9d38-b8b11f4908a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:31.318608 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578b83d-835a-47c9-9d38-b8b11f4908a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:31.318874 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578b83d-835a-47c9-9d38-b8b11f4908a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:31.328380 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578b83d-835a-47c9-9d38-b8b11f4908a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:31.329823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578b83d-835a-47c9-9d38-b8b11f4908a2 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:31.340513 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1412/2817] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:08:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:08:34.454924 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29365a35-d0ab-4509-ab3f-8a178996886a service placement] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:34.466089 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29365a35-d0ab-4509-ab3f-8a178996886a service placement] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:34.476585 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94794dfb-bfb6-4606-916f-f8b30965ec6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:34.478864 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94794dfb-bfb6-4606-916f-f8b30965ec6f service placement] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:34.479159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94794dfb-bfb6-4606-916f-f8b30965ec6f service placement] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:34.479510 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94794dfb-bfb6-4606-916f-f8b30965ec6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:34.481649 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-94794dfb-bfb6-4606-916f-f8b30965ec6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:08:34.496109 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94794dfb-bfb6-4606-916f-f8b30965ec6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:34.497832 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94794dfb-bfb6-4606-916f-f8b30965ec6f service placement] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:34.509269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1406/2818] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:08:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:08:34.522685 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578b83d-835a-47c9-9d38-b8b11f4908a2 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:34.534400 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578b83d-835a-47c9-9d38-b8b11f4908a2 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:34.552123 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa872f-d740-489e-9430-36f014f6ba05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:34.553375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa872f-d740-489e-9430-36f014f6ba05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:34.553860 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa872f-d740-489e-9430-36f014f6ba05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:34.554233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa872f-d740-489e-9430-36f014f6ba05 admin admin] PATH_INFO: `/v3/users/631157435dd242be80eb9f9fbf89ae76` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:34.555683 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa872f-d740-489e-9430-36f014f6ba05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=631157435dd242be80eb9f9fbf89ae76)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:34.557916 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa872f-d740-489e-9430-36f014f6ba05 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:34.613444 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8faa872f-d740-489e-9430-36f014f6ba05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '631157435dd242be80eb9f9fbf89ae76'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:34.630773 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8faa872f-d740-489e-9430-36f014f6ba05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '631157435dd242be80eb9f9fbf89ae76'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:34.639697 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8faa872f-d740-489e-9430-36f014f6ba05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '631157435dd242be80eb9f9fbf89ae76'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:34.640768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8faa872f-d740-489e-9430-36f014f6ba05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '631157435dd242be80eb9f9fbf89ae76'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:34.641537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8faa872f-d740-489e-9430-36f014f6ba05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '631157435dd242be80eb9f9fbf89ae76'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:34.664999 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1413/2819] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 06:08:34 2022] DELETE /identity/v3/users/631157435dd242be80eb9f9fbf89ae76 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:08:34.862773 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94794dfb-bfb6-4606-916f-f8b30965ec6f service placement] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:34.875732 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94794dfb-bfb6-4606-916f-f8b30965ec6f service placement] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:34.936105 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e825fb6-a952-4104-8e43-d68ac3403f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:34.936934 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e825fb6-a952-4104-8e43-d68ac3403f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:34.937214 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e825fb6-a952-4104-8e43-d68ac3403f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:34.937483 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e825fb6-a952-4104-8e43-d68ac3403f42 admin admin] PATH_INFO: `/v3/projects/1c87efe1b2b74d7d8749ae5e14bdd412` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:34.938765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e825fb6-a952-4104-8e43-d68ac3403f42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c87efe1b2b74d7d8749ae5e14bdd412)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:34.940772 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e825fb6-a952-4104-8e43-d68ac3403f42 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:34.989652 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-7e825fb6-a952-4104-8e43-d68ac3403f42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c87efe1b2b74d7d8749ae5e14bdd412', 'request_id': 'req-7e825fb6-a952-4104-8e43-d68ac3403f42'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:34.996004 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-7e825fb6-a952-4104-8e43-d68ac3403f42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c87efe1b2b74d7d8749ae5e14bdd412', 'request_id': 'req-7e825fb6-a952-4104-8e43-d68ac3403f42'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:35.002139 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-7e825fb6-a952-4104-8e43-d68ac3403f42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c87efe1b2b74d7d8749ae5e14bdd412', 'request_id': 'req-7e825fb6-a952-4104-8e43-d68ac3403f42'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:35.016235 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-7e825fb6-a952-4104-8e43-d68ac3403f42 admin admin] The token cache is being invalidate because project 1c87efe1b2b74d7d8749ae5e14bdd412 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:08:35.016525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-7e825fb6-a952-4104-8e43-d68ac3403f42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:35.017155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-7e825fb6-a952-4104-8e43-d68ac3403f42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:35.018731 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1407/2820] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 06:08:34 2022] DELETE /identity/v3/projects/1c87efe1b2b74d7d8749ae5e14bdd412 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:08:35.023842 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021dcbc-1c66-4e70-b545-0f21bcfe8099 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:35.024136 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021dcbc-1c66-4e70-b545-0f21bcfe8099 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:35.024415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021dcbc-1c66-4e70-b545-0f21bcfe8099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:35.043512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-0021dcbc-1c66-4e70-b545-0f21bcfe8099 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:08:35.044704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-0021dcbc-1c66-4e70-b545-0f21bcfe8099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:08:35.110373 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1414/2821] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:08:35.121554 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac66f1-039b-4699-af33-0a6c26eebd29 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:35.121852 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac66f1-039b-4699-af33-0a6c26eebd29 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:35.122129 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac66f1-039b-4699-af33-0a6c26eebd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:35.143487 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-e0ac66f1-039b-4699-af33-0a6c26eebd29 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:08:35.144715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-e0ac66f1-039b-4699-af33-0a6c26eebd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:08:35.154438 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1408/2822] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:08:35.159541 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021dcbc-1c66-4e70-b545-0f21bcfe8099 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:35.170026 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021dcbc-1c66-4e70-b545-0f21bcfe8099 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:35.175139 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56208396-3e88-49f8-838f-45eff019f910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:35.175787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56208396-3e88-49f8-838f-45eff019f910 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:35.176066 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56208396-3e88-49f8-838f-45eff019f910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:35.176338 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56208396-3e88-49f8-838f-45eff019f910 admin admin] PATH_INFO: `/v3/domains` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:35.177064 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56208396-3e88-49f8-838f-45eff019f910 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:08:35.177368 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56208396-3e88-49f8-838f-45eff019f910 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:35.178967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56208396-3e88-49f8-838f-45eff019f910 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:35.191866 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1415/2823] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 06:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:08:35.196066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac66f1-039b-4699-af33-0a6c26eebd29 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:35.209501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac66f1-039b-4699-af33-0a6c26eebd29 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:35.216780 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e06476-446e-4acc-b409-ea32e6cd3af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:35.217782 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e06476-446e-4acc-b409-ea32e6cd3af8 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:35.218181 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e06476-446e-4acc-b409-ea32e6cd3af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:35.218567 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e06476-446e-4acc-b409-ea32e6cd3af8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:35.219691 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e06476-446e-4acc-b409-ea32e6cd3af8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:35.222699 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e06476-446e-4acc-b409-ea32e6cd3af8 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:35.262172 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1409/2824] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 06:08:35 2022] POST /identity/v3/projects => generated 365 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:08:35.266613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56208396-3e88-49f8-838f-45eff019f910 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:35.276875 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56208396-3e88-49f8-838f-45eff019f910 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:35.323692 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:08:35.338862 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf15f4c-6ca6-4153-98c7-7f8d9f7c9111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:35.339561 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf15f4c-6ca6-4153-98c7-7f8d9f7c9111 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:35.339845 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf15f4c-6ca6-4153-98c7-7f8d9f7c9111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:35.340120 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf15f4c-6ca6-4153-98c7-7f8d9f7c9111 admin admin] PATH_INFO: `/v3/users` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:35.340825 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf15f4c-6ca6-4153-98c7-7f8d9f7c9111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:35.342787 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf15f4c-6ca6-4153-98c7-7f8d9f7c9111 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:35.364474 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1416/2825] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 06:08:35 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:08:35.368869 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e06476-446e-4acc-b409-ea32e6cd3af8 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:35.379572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e06476-446e-4acc-b409-ea32e6cd3af8 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:35.384749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820dd0e-18a8-4574-88ad-dbfbbff02528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:35.385406 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820dd0e-18a8-4574-88ad-dbfbbff02528 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:35.385687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820dd0e-18a8-4574-88ad-dbfbbff02528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:35.385962 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820dd0e-18a8-4574-88ad-dbfbbff02528 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:35.386680 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820dd0e-18a8-4574-88ad-dbfbbff02528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:35.388461 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820dd0e-18a8-4574-88ad-dbfbbff02528 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:35.398545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1410/2826] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 06:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:08:35.402528 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf15f4c-6ca6-4153-98c7-7f8d9f7c9111 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:35.412893 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf15f4c-6ca6-4153-98c7-7f8d9f7c9111 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:35.417681 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd7df5-a20c-4dd9-bf20-2c79e5c85db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:35.418330 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd7df5-a20c-4dd9-bf20-2c79e5c85db9 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:35.418606 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd7df5-a20c-4dd9-bf20-2c79e5c85db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:35.418861 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd7df5-a20c-4dd9-bf20-2c79e5c85db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:35.419623 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd7df5-a20c-4dd9-bf20-2c79e5c85db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:35.421219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd7df5-a20c-4dd9-bf20-2c79e5c85db9 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:35.433616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1417/2827] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 06:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 21 06:08:35.439544 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820dd0e-18a8-4574-88ad-dbfbbff02528 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:35.450234 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820dd0e-18a8-4574-88ad-dbfbbff02528 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:35.456302 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500392d9-7f93-4014-aa6c-ac21edd246f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:35.457198 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500392d9-7f93-4014-aa6c-ac21edd246f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:35.457517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500392d9-7f93-4014-aa6c-ac21edd246f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:35.457797 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500392d9-7f93-4014-aa6c-ac21edd246f5 admin admin] PATH_INFO: `/v3/projects/022cf475b17f487c80c95b164f9e2793/users/12441540cfbb4919900e6ea6479ec55b/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:35.473992 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500392d9-7f93-4014-aa6c-ac21edd246f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=022cf475b17f487c80c95b164f9e2793, user_id=12441540cfbb4919900e6ea6479ec55b, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:35.476838 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500392d9-7f93-4014-aa6c-ac21edd246f5 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:35.492431 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1411/2828] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 06:08:35 2022] PUT /identity/v3/projects/022cf475b17f487c80c95b164f9e2793/users/12441540cfbb4919900e6ea6479ec55b/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:08:35.504811 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0389b-2ddc-4fa9-9d08-aaa604bbb8d7 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:35.505186 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0389b-2ddc-4fa9-9d08-aaa604bbb8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:35.505574 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0389b-2ddc-4fa9-9d08-aaa604bbb8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:35.540263 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-bfb0389b-2ddc-4fa9-9d08-aaa604bbb8d7 None None] MFA Rules not processed for user `12441540cfbb4919900e6ea6479ec55b`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:08:35.541469 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-bfb0389b-2ddc-4fa9-9d08-aaa604bbb8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:08:35.608987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1418/2829] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:08:35 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:08:35.621701 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500392d9-7f93-4014-aa6c-ac21edd246f5 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:35.623575 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-500392d9-7f93-4014-aa6c-ac21edd246f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:08:35.628576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:08:35.707298 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500392d9-7f93-4014-aa6c-ac21edd246f5 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:35.711356 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9f108-d6c9-4277-a506-d34d6497cf65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:35.712005 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9f108-d6c9-4277-a506-d34d6497cf65 service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:35.712272 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9f108-d6c9-4277-a506-d34d6497cf65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:35.712575 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9f108-d6c9-4277-a506-d34d6497cf65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:35.721367 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9f108-d6c9-4277-a506-d34d6497cf65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:35.722821 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9f108-d6c9-4277-a506-d34d6497cf65 service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:35.732052 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1412/2830] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:08:35 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:08:40.126232 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0389b-2ddc-4fa9-9d08-aaa604bbb8d7 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:40.127843 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-bfb0389b-2ddc-4fa9-9d08-aaa604bbb8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:08:40.143538 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0389b-2ddc-4fa9-9d08-aaa604bbb8d7 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:40.148081 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c071615-3f80-4a66-a322-da7af630d393 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:40.148731 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c071615-3f80-4a66-a322-da7af630d393 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:40.149019 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c071615-3f80-4a66-a322-da7af630d393 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:40.149294 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c071615-3f80-4a66-a322-da7af630d393 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:40.151178 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6c071615-3f80-4a66-a322-da7af630d393 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:08:40.226630 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c071615-3f80-4a66-a322-da7af630d393 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:40.228057 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c071615-3f80-4a66-a322-da7af630d393 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:40.239295 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1419/2831] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:08:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:08:44.529877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9f108-d6c9-4277-a506-d34d6497cf65 service cinder] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:44.595658 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9f108-d6c9-4277-a506-d34d6497cf65 service cinder] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:44.600606 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c4b29-e3d9-4c41-8c42-fd2ecf541e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:44.601580 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c4b29-e3d9-4c41-8c42-fd2ecf541e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:44.601762 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c4b29-e3d9-4c41-8c42-fd2ecf541e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:44.602063 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c4b29-e3d9-4c41-8c42-fd2ecf541e55 admin admin] PATH_INFO: `/v3/users/12441540cfbb4919900e6ea6479ec55b` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:44.603365 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c4b29-e3d9-4c41-8c42-fd2ecf541e55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12441540cfbb4919900e6ea6479ec55b)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:44.605298 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c4b29-e3d9-4c41-8c42-fd2ecf541e55 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:44.657943 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-bb8c4b29-e3d9-4c41-8c42-fd2ecf541e55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12441540cfbb4919900e6ea6479ec55b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:44.672693 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-bb8c4b29-e3d9-4c41-8c42-fd2ecf541e55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12441540cfbb4919900e6ea6479ec55b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:44.679738 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-bb8c4b29-e3d9-4c41-8c42-fd2ecf541e55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12441540cfbb4919900e6ea6479ec55b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:44.680557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-bb8c4b29-e3d9-4c41-8c42-fd2ecf541e55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12441540cfbb4919900e6ea6479ec55b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:44.681324 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-bb8c4b29-e3d9-4c41-8c42-fd2ecf541e55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12441540cfbb4919900e6ea6479ec55b'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:44.694910 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1413/2832] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 06:08:44 2022] DELETE /identity/v3/users/12441540cfbb4919900e6ea6479ec55b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:08:44.700582 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b7b5f-28d5-4eb8-9034-cdbf5168427d None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:44.701182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b7b5f-28d5-4eb8-9034-cdbf5168427d None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:44.701182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b7b5f-28d5-4eb8-9034-cdbf5168427d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:44.720002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-cb4b7b5f-28d5-4eb8-9034-cdbf5168427d None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:08:44.721239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-cb4b7b5f-28d5-4eb8-9034-cdbf5168427d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:08:44.756328 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:08:44.784103 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1420/2833] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:08:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:08:44.794220 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c4b29-e3d9-4c41-8c42-fd2ecf541e55 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:44.795967 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-bb8c4b29-e3d9-4c41-8c42-fd2ecf541e55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:08:44.806517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:08:44.859820 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c4b29-e3d9-4c41-8c42-fd2ecf541e55 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:44.863608 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a84d03-183c-4f4f-8cf2-accaaa6fbca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:44.864439 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a84d03-183c-4f4f-8cf2-accaaa6fbca5 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:44.864776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a84d03-183c-4f4f-8cf2-accaaa6fbca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:44.865062 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a84d03-183c-4f4f-8cf2-accaaa6fbca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:44.875112 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a84d03-183c-4f4f-8cf2-accaaa6fbca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:44.876569 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a84d03-183c-4f4f-8cf2-accaaa6fbca5 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:44.886735 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1414/2834] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:08:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:08:45.162540 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b7b5f-28d5-4eb8-9034-cdbf5168427d None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:45.164715 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-cb4b7b5f-28d5-4eb8-9034-cdbf5168427d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:08:45.179995 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b7b5f-28d5-4eb8-9034-cdbf5168427d None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:45.184853 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212b652-0e6d-4a86-8af8-eedfb7509cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:45.185727 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212b652-0e6d-4a86-8af8-eedfb7509cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:45.186262 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212b652-0e6d-4a86-8af8-eedfb7509cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:45.186548 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212b652-0e6d-4a86-8af8-eedfb7509cd7 admin admin] PATH_INFO: `/v3/projects/022cf475b17f487c80c95b164f9e2793` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:45.187964 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212b652-0e6d-4a86-8af8-eedfb7509cd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=022cf475b17f487c80c95b164f9e2793)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:45.189827 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212b652-0e6d-4a86-8af8-eedfb7509cd7 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:45.238517 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6212b652-0e6d-4a86-8af8-eedfb7509cd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '022cf475b17f487c80c95b164f9e2793', 'request_id': 'req-6212b652-0e6d-4a86-8af8-eedfb7509cd7'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:45.244306 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6212b652-0e6d-4a86-8af8-eedfb7509cd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '022cf475b17f487c80c95b164f9e2793', 'request_id': 'req-6212b652-0e6d-4a86-8af8-eedfb7509cd7'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:45.252121 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6212b652-0e6d-4a86-8af8-eedfb7509cd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '022cf475b17f487c80c95b164f9e2793', 'request_id': 'req-6212b652-0e6d-4a86-8af8-eedfb7509cd7'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:45.263677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6212b652-0e6d-4a86-8af8-eedfb7509cd7 admin admin] The token cache is being invalidate because project 022cf475b17f487c80c95b164f9e2793 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:08:45.264064 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6212b652-0e6d-4a86-8af8-eedfb7509cd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:45.264790 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6212b652-0e6d-4a86-8af8-eedfb7509cd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:08:45.266518 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1421/2835] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 06:08:45 2022] DELETE /identity/v3/projects/022cf475b17f487c80c95b164f9e2793 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:08:45.271917 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10127ed-bd98-41e2-92c3-19c4a0bcfd9c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:45.272305 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10127ed-bd98-41e2-92c3-19c4a0bcfd9c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:45.272605 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10127ed-bd98-41e2-92c3-19c4a0bcfd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:45.290624 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-a10127ed-bd98-41e2-92c3-19c4a0bcfd9c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:08:45.291799 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a10127ed-bd98-41e2-92c3-19c4a0bcfd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:08:45.355620 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1415/2836] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:08:45.363864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd4b86-8d48-474c-afea-3df9fd73f0b5 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:45.364156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd4b86-8d48-474c-afea-3df9fd73f0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:45.364436 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd4b86-8d48-474c-afea-3df9fd73f0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:45.383735 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-07dd4b86-8d48-474c-afea-3df9fd73f0b5 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:08:45.384940 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-07dd4b86-8d48-474c-afea-3df9fd73f0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:08:45.397511 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1422/2837] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:08:45.403732 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10127ed-bd98-41e2-92c3-19c4a0bcfd9c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:45.414781 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10127ed-bd98-41e2-92c3-19c4a0bcfd9c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:45.421299 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0389fd8-1bd2-48b8-9900-8b845f5d00bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:45.421961 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0389fd8-1bd2-48b8-9900-8b845f5d00bb admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:45.422248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0389fd8-1bd2-48b8-9900-8b845f5d00bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:45.422540 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0389fd8-1bd2-48b8-9900-8b845f5d00bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:45.423333 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0389fd8-1bd2-48b8-9900-8b845f5d00bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:08:45.423625 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0389fd8-1bd2-48b8-9900-8b845f5d00bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:45.425306 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0389fd8-1bd2-48b8-9900-8b845f5d00bb admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:45.437269 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1416/2838] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 06:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 21 06:08:45.441576 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd4b86-8d48-474c-afea-3df9fd73f0b5 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:45.450912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd4b86-8d48-474c-afea-3df9fd73f0b5 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:45.456198 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf37f1b-7ebf-4080-93a3-d93fa88a0299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:45.456847 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf37f1b-7ebf-4080-93a3-d93fa88a0299 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:45.457130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf37f1b-7ebf-4080-93a3-d93fa88a0299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:45.457405 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf37f1b-7ebf-4080-93a3-d93fa88a0299 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:45.458091 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf37f1b-7ebf-4080-93a3-d93fa88a0299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:45.460042 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf37f1b-7ebf-4080-93a3-d93fa88a0299 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:45.488219 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1423/2839] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 06:08:45 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:08:45.492418 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0389fd8-1bd2-48b8-9900-8b845f5d00bb admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:45.502429 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0389fd8-1bd2-48b8-9900-8b845f5d00bb admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:45.558931 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eac492-a886-48c1-94a8-19720f5c8fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:45.559616 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eac492-a886-48c1-94a8-19720f5c8fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:45.559825 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eac492-a886-48c1-94a8-19720f5c8fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:45.560102 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eac492-a886-48c1-94a8-19720f5c8fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:45.560811 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eac492-a886-48c1-94a8-19720f5c8fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:45.562743 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eac492-a886-48c1-94a8-19720f5c8fe2 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:45.585909 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1417/2840] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 06:08:45 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:08:45.590032 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf37f1b-7ebf-4080-93a3-d93fa88a0299 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:45.600925 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf37f1b-7ebf-4080-93a3-d93fa88a0299 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:45.606193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c0170-4f19-44a2-9197-538489651265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:45.606852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c0170-4f19-44a2-9197-538489651265 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:45.607164 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c0170-4f19-44a2-9197-538489651265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:45.607428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c0170-4f19-44a2-9197-538489651265 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:45.608258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c0170-4f19-44a2-9197-538489651265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:45.609932 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c0170-4f19-44a2-9197-538489651265 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:45.620851 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1424/2841] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 06:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:08:45.625382 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eac492-a886-48c1-94a8-19720f5c8fe2 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:45.636121 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eac492-a886-48c1-94a8-19720f5c8fe2 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:45.641415 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735e1ef-b5f2-4d3e-90a0-b299b84fd758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:45.642077 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735e1ef-b5f2-4d3e-90a0-b299b84fd758 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:45.642363 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735e1ef-b5f2-4d3e-90a0-b299b84fd758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:45.642643 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735e1ef-b5f2-4d3e-90a0-b299b84fd758 admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:45.643402 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735e1ef-b5f2-4d3e-90a0-b299b84fd758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:45.645008 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735e1ef-b5f2-4d3e-90a0-b299b84fd758 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:45.655970 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1418/2842] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 06:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:08:45.660371 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c0170-4f19-44a2-9197-538489651265 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:45.670836 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c0170-4f19-44a2-9197-538489651265 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:45.676031 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99473148-cbdf-4de4-8f2a-4aa2c65b7407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:45.676904 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99473148-cbdf-4de4-8f2a-4aa2c65b7407 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:45.677182 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99473148-cbdf-4de4-8f2a-4aa2c65b7407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:45.677453 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99473148-cbdf-4de4-8f2a-4aa2c65b7407 admin admin] PATH_INFO: `/v3/projects/1785950d0d244a26acc89ed5556d3be9/users/09d1e1699bdc454aa04b9681d854ebe9/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:45.693035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99473148-cbdf-4de4-8f2a-4aa2c65b7407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1785950d0d244a26acc89ed5556d3be9, user_id=09d1e1699bdc454aa04b9681d854ebe9, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:45.696000 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99473148-cbdf-4de4-8f2a-4aa2c65b7407 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:45.711673 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1425/2843] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 06:08:45 2022] PUT /identity/v3/projects/1785950d0d244a26acc89ed5556d3be9/users/09d1e1699bdc454aa04b9681d854ebe9/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:08:45.722208 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c14241-0a0a-4564-8049-597faac387d7 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:45.722479 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c14241-0a0a-4564-8049-597faac387d7 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:45.722755 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c14241-0a0a-4564-8049-597faac387d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:45.749086 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-b4c14241-0a0a-4564-8049-597faac387d7 None None] MFA Rules not processed for user `09d1e1699bdc454aa04b9681d854ebe9`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:08:45.750288 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-b4c14241-0a0a-4564-8049-597faac387d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:08:45.754322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:08:45.817848 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1419/2844] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:08:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:08:45.822295 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99473148-cbdf-4de4-8f2a-4aa2c65b7407 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:45.886276 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99473148-cbdf-4de4-8f2a-4aa2c65b7407 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:45.891852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb688ef4-4284-40dd-bd00-e4003d811436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:45.892667 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb688ef4-4284-40dd-bd00-e4003d811436 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:45.892954 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb688ef4-4284-40dd-bd00-e4003d811436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:45.893222 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb688ef4-4284-40dd-bd00-e4003d811436 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:45.893912 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb688ef4-4284-40dd-bd00-e4003d811436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:45.895891 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb688ef4-4284-40dd-bd00-e4003d811436 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:45.899264 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:08:45.925596 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1426/2845] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 06:08:45 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:08:45.929897 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c14241-0a0a-4564-8049-597faac387d7 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:45.941170 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c14241-0a0a-4564-8049-597faac387d7 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:46.009541 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab4d40-b56d-4a21-ae98-b7d60399d605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:46.010218 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab4d40-b56d-4a21-ae98-b7d60399d605 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:46.010505 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab4d40-b56d-4a21-ae98-b7d60399d605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:46.010787 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab4d40-b56d-4a21-ae98-b7d60399d605 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:46.011528 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab4d40-b56d-4a21-ae98-b7d60399d605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:46.013486 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab4d40-b56d-4a21-ae98-b7d60399d605 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:46.038708 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1420/2846] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 06:08:45 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:08:46.042848 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb688ef4-4284-40dd-bd00-e4003d811436 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:46.053544 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb688ef4-4284-40dd-bd00-e4003d811436 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:46.058691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61c8ad-de51-434d-a982-19c11e2ab5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:46.059375 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61c8ad-de51-434d-a982-19c11e2ab5ea admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:46.059656 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61c8ad-de51-434d-a982-19c11e2ab5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:46.059930 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61c8ad-de51-434d-a982-19c11e2ab5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:46.060660 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61c8ad-de51-434d-a982-19c11e2ab5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:46.062268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61c8ad-de51-434d-a982-19c11e2ab5ea admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:46.073850 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1427/2847] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 06:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:08:46.078744 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab4d40-b56d-4a21-ae98-b7d60399d605 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:46.089973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab4d40-b56d-4a21-ae98-b7d60399d605 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:46.095263 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da97d4-7390-4cfc-a941-6f31bfca1afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:46.096159 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da97d4-7390-4cfc-a941-6f31bfca1afe admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:46.096445 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da97d4-7390-4cfc-a941-6f31bfca1afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:46.096726 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da97d4-7390-4cfc-a941-6f31bfca1afe admin admin] PATH_INFO: `/v3/projects/45561bee9e0542f48c002451991de44b/users/5d285f40ae9e40ca804e75bb1bf72fc0/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:46.112041 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da97d4-7390-4cfc-a941-6f31bfca1afe admin admin] RBAC: Authorizing `identity:create_grant(project_id=45561bee9e0542f48c002451991de44b, user_id=5d285f40ae9e40ca804e75bb1bf72fc0, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:46.114735 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da97d4-7390-4cfc-a941-6f31bfca1afe admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:46.129204 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1421/2848] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 06:08:46 2022] PUT /identity/v3/projects/45561bee9e0542f48c002451991de44b/users/5d285f40ae9e40ca804e75bb1bf72fc0/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:08:46.133235 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61c8ad-de51-434d-a982-19c11e2ab5ea admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:46.143655 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61c8ad-de51-434d-a982-19c11e2ab5ea admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:46.201107 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1fdd3-19fb-415b-8ad0-9d574c07d8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:46.201852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1fdd3-19fb-415b-8ad0-9d574c07d8b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:46.202130 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1fdd3-19fb-415b-8ad0-9d574c07d8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:46.202401 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1fdd3-19fb-415b-8ad0-9d574c07d8b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:46.203164 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d1fdd3-19fb-415b-8ad0-9d574c07d8b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:46.204775 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d1fdd3-19fb-415b-8ad0-9d574c07d8b5 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:46.214445 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1428/2849] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 06:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:08:46.219720 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da97d4-7390-4cfc-a941-6f31bfca1afe admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:46.235336 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da97d4-7390-4cfc-a941-6f31bfca1afe admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:46.240207 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a07a54-5549-4a9c-ae83-947083aef2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:46.241100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a07a54-5549-4a9c-ae83-947083aef2c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:46.241377 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a07a54-5549-4a9c-ae83-947083aef2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:46.241655 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a07a54-5549-4a9c-ae83-947083aef2c4 admin admin] PATH_INFO: `/v3/projects/45561bee9e0542f48c002451991de44b/users/5d285f40ae9e40ca804e75bb1bf72fc0/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:46.243665 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a07a54-5549-4a9c-ae83-947083aef2c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45561bee9e0542f48c002451991de44b, user_id=5d285f40ae9e40ca804e75bb1bf72fc0, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:46.246316 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a07a54-5549-4a9c-ae83-947083aef2c4 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:46.258072 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1422/2850] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 06:08:46 2022] PUT /identity/v3/projects/45561bee9e0542f48c002451991de44b/users/5d285f40ae9e40ca804e75bb1bf72fc0/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:08:46.269830 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360c169-e5f4-45a7-836e-cceb5dc599cb None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:46.270125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360c169-e5f4-45a7-836e-cceb5dc599cb None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:46.270409 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360c169-e5f4-45a7-836e-cceb5dc599cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:46.306125 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-4360c169-e5f4-45a7-836e-cceb5dc599cb None None] MFA Rules not processed for user `5d285f40ae9e40ca804e75bb1bf72fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:08:46.308983 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4360c169-e5f4-45a7-836e-cceb5dc599cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:08:46.402092 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1429/2851] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:08:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:08:46.414066 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a07a54-5549-4a9c-ae83-947083aef2c4 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:46.416198 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-95a07a54-5549-4a9c-ae83-947083aef2c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:08:46.476494 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a07a54-5549-4a9c-ae83-947083aef2c4 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:46.480348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951acd60-de8e-4c4b-b383-1469a07be8d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:46.480994 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951acd60-de8e-4c4b-b383-1469a07be8d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:46.481273 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951acd60-de8e-4c4b-b383-1469a07be8d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:46.481549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951acd60-de8e-4c4b-b383-1469a07be8d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:46.491025 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:08:46.539529 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951acd60-de8e-4c4b-b383-1469a07be8d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:46.540973 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951acd60-de8e-4c4b-b383-1469a07be8d8 service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:46.550689 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1423/2852] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:08:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:08:52.237241 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360c169-e5f4-45a7-836e-cceb5dc599cb None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:08:52.239448 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4360c169-e5f4-45a7-836e-cceb5dc599cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:08:52.260143 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360c169-e5f4-45a7-836e-cceb5dc599cb None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:08:52.266616 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5720abc-2df6-4b8d-91b0-3096a986cf60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:08:52.267321 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5720abc-2df6-4b8d-91b0-3096a986cf60 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:08:52.267611 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5720abc-2df6-4b8d-91b0-3096a986cf60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:08:52.267890 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5720abc-2df6-4b8d-91b0-3096a986cf60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:08:52.278268 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5720abc-2df6-4b8d-91b0-3096a986cf60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:08:52.279795 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5720abc-2df6-4b8d-91b0-3096a986cf60 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:08:52.290633 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1430/2853] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:08:52 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:08:56.546104 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=22 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:02.286151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=80,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:20.482735 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951acd60-de8e-4c4b-b383-1469a07be8d8 service cinder] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:20.551598 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951acd60-de8e-4c4b-b383-1469a07be8d8 service cinder] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:20.556679 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b73599-e485-4da0-91f1-d7f7822ded17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:20.557557 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b73599-e485-4da0-91f1-d7f7822ded17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:20.557836 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b73599-e485-4da0-91f1-d7f7822ded17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:20.558116 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b73599-e485-4da0-91f1-d7f7822ded17 admin admin] PATH_INFO: `/v3/users/09d1e1699bdc454aa04b9681d854ebe9` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:20.559413 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b73599-e485-4da0-91f1-d7f7822ded17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09d1e1699bdc454aa04b9681d854ebe9)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:20.561359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b73599-e485-4da0-91f1-d7f7822ded17 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:20.610365 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-62b73599-e485-4da0-91f1-d7f7822ded17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09d1e1699bdc454aa04b9681d854ebe9'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:20.622527 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-62b73599-e485-4da0-91f1-d7f7822ded17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09d1e1699bdc454aa04b9681d854ebe9'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:20.629236 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-62b73599-e485-4da0-91f1-d7f7822ded17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09d1e1699bdc454aa04b9681d854ebe9'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:20.629880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-62b73599-e485-4da0-91f1-d7f7822ded17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09d1e1699bdc454aa04b9681d854ebe9'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:20.630463 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-62b73599-e485-4da0-91f1-d7f7822ded17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09d1e1699bdc454aa04b9681d854ebe9'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:20.644327 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1424/2854] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 06:09:20 2022] DELETE /identity/v3/users/09d1e1699bdc454aa04b9681d854ebe9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:09:20.649085 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ceb0d-d0fa-42e4-8e62-0e8f6fed98d8 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:20.649621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ceb0d-d0fa-42e4-8e62-0e8f6fed98d8 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:20.649621 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ceb0d-d0fa-42e4-8e62-0e8f6fed98d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:20.668466 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-370ceb0d-d0fa-42e4-8e62-0e8f6fed98d8 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:20.669687 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-370ceb0d-d0fa-42e4-8e62-0e8f6fed98d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:20.735512 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1431/2855] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:20.749176 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b73599-e485-4da0-91f1-d7f7822ded17 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:20.750944 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-62b73599-e485-4da0-91f1-d7f7822ded17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:20.813010 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b73599-e485-4da0-91f1-d7f7822ded17 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:20.817566 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04876c5c-9041-4e3f-b8a8-481ffddbf4f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:20.818221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04876c5c-9041-4e3f-b8a8-481ffddbf4f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:20.818504 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04876c5c-9041-4e3f-b8a8-481ffddbf4f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:20.818780 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04876c5c-9041-4e3f-b8a8-481ffddbf4f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:20.828549 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04876c5c-9041-4e3f-b8a8-481ffddbf4f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:20.829988 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04876c5c-9041-4e3f-b8a8-481ffddbf4f9 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:20.840001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1425/2856] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:09:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:09:21.118252 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ceb0d-d0fa-42e4-8e62-0e8f6fed98d8 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:21.119729 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-370ceb0d-d0fa-42e4-8e62-0e8f6fed98d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:21.134764 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ceb0d-d0fa-42e4-8e62-0e8f6fed98d8 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:21.139791 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c063056-2a7a-433c-8604-48a0ded0bb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:21.140677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c063056-2a7a-433c-8604-48a0ded0bb36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:21.140952 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c063056-2a7a-433c-8604-48a0ded0bb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:21.141226 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c063056-2a7a-433c-8604-48a0ded0bb36 admin admin] PATH_INFO: `/v3/projects/1785950d0d244a26acc89ed5556d3be9` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:21.142543 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c063056-2a7a-433c-8604-48a0ded0bb36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1785950d0d244a26acc89ed5556d3be9)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:21.144543 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c063056-2a7a-433c-8604-48a0ded0bb36 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:21.196924 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6c063056-2a7a-433c-8604-48a0ded0bb36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1785950d0d244a26acc89ed5556d3be9', 'request_id': 'req-6c063056-2a7a-433c-8604-48a0ded0bb36'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:21.202716 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6c063056-2a7a-433c-8604-48a0ded0bb36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1785950d0d244a26acc89ed5556d3be9', 'request_id': 'req-6c063056-2a7a-433c-8604-48a0ded0bb36'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:21.212127 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6c063056-2a7a-433c-8604-48a0ded0bb36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1785950d0d244a26acc89ed5556d3be9', 'request_id': 'req-6c063056-2a7a-433c-8604-48a0ded0bb36'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:21.223907 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6c063056-2a7a-433c-8604-48a0ded0bb36 admin admin] The token cache is being invalidate because project 1785950d0d244a26acc89ed5556d3be9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:09:21.224196 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6c063056-2a7a-433c-8604-48a0ded0bb36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:21.224823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-6c063056-2a7a-433c-8604-48a0ded0bb36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:21.226372 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1432/2857] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 06:09:21 2022] DELETE /identity/v3/projects/1785950d0d244a26acc89ed5556d3be9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:09:21.230564 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04876c5c-9041-4e3f-b8a8-481ffddbf4f9 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:21.240602 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04876c5c-9041-4e3f-b8a8-481ffddbf4f9 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:21.260376 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:21.301683 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ff0ab-386e-4f58-a8b0-aaaecaa96391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:21.302564 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ff0ab-386e-4f58-a8b0-aaaecaa96391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:21.302849 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ff0ab-386e-4f58-a8b0-aaaecaa96391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:21.303160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ff0ab-386e-4f58-a8b0-aaaecaa96391 admin admin] PATH_INFO: `/v3/users/5d285f40ae9e40ca804e75bb1bf72fc0` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:21.304420 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ff0ab-386e-4f58-a8b0-aaaecaa96391 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d285f40ae9e40ca804e75bb1bf72fc0)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:21.306376 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ff0ab-386e-4f58-a8b0-aaaecaa96391 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:21.355889 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-671ff0ab-386e-4f58-a8b0-aaaecaa96391 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d285f40ae9e40ca804e75bb1bf72fc0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:21.367145 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-671ff0ab-386e-4f58-a8b0-aaaecaa96391 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d285f40ae9e40ca804e75bb1bf72fc0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:21.373865 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-671ff0ab-386e-4f58-a8b0-aaaecaa96391 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d285f40ae9e40ca804e75bb1bf72fc0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:21.374642 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-671ff0ab-386e-4f58-a8b0-aaaecaa96391 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d285f40ae9e40ca804e75bb1bf72fc0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:21.375379 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-671ff0ab-386e-4f58-a8b0-aaaecaa96391 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d285f40ae9e40ca804e75bb1bf72fc0'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:21.390401 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1426/2858] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 06:09:21 2022] DELETE /identity/v3/users/5d285f40ae9e40ca804e75bb1bf72fc0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:09:21.669987 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c063056-2a7a-433c-8604-48a0ded0bb36 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:21.671568 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-6c063056-2a7a-433c-8604-48a0ded0bb36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:21.686467 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c063056-2a7a-433c-8604-48a0ded0bb36 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:21.744704 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9a45e-1fbe-4eb6-85db-c685d99f9ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:21.745601 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9a45e-1fbe-4eb6-85db-c685d99f9ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:21.745944 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9a45e-1fbe-4eb6-85db-c685d99f9ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:21.746216 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9a45e-1fbe-4eb6-85db-c685d99f9ca5 admin admin] PATH_INFO: `/v3/projects/45561bee9e0542f48c002451991de44b` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:21.747440 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9a45e-1fbe-4eb6-85db-c685d99f9ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45561bee9e0542f48c002451991de44b)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:21.749358 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9a45e-1fbe-4eb6-85db-c685d99f9ca5 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:21.780380 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:21.798002 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-05c9a45e-1fbe-4eb6-85db-c685d99f9ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45561bee9e0542f48c002451991de44b', 'request_id': 'req-05c9a45e-1fbe-4eb6-85db-c685d99f9ca5'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:21.804187 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-05c9a45e-1fbe-4eb6-85db-c685d99f9ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45561bee9e0542f48c002451991de44b', 'request_id': 'req-05c9a45e-1fbe-4eb6-85db-c685d99f9ca5'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:21.813062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-05c9a45e-1fbe-4eb6-85db-c685d99f9ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45561bee9e0542f48c002451991de44b', 'request_id': 'req-05c9a45e-1fbe-4eb6-85db-c685d99f9ca5'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:21.825036 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-05c9a45e-1fbe-4eb6-85db-c685d99f9ca5 admin admin] The token cache is being invalidate because project 45561bee9e0542f48c002451991de44b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:09:21.825329 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-05c9a45e-1fbe-4eb6-85db-c685d99f9ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:21.825982 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-05c9a45e-1fbe-4eb6-85db-c685d99f9ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:21.827603 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1433/2859] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 06:09:21 2022] DELETE /identity/v3/projects/45561bee9e0542f48c002451991de44b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:09:21.832576 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18f1d0-653c-47c7-befa-ff2ffff31571 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:21.833049 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18f1d0-653c-47c7-befa-ff2ffff31571 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:21.833155 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18f1d0-653c-47c7-befa-ff2ffff31571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:21.851209 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-fd18f1d0-653c-47c7-befa-ff2ffff31571 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:21.852987 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-fd18f1d0-653c-47c7-befa-ff2ffff31571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:21.918381 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1427/2860] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:21.926353 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0452c-dfa6-4ee4-a8d1-d440506c3784 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:21.926638 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0452c-dfa6-4ee4-a8d1-d440506c3784 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:21.926877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0452c-dfa6-4ee4-a8d1-d440506c3784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:21.945938 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-7ce0452c-dfa6-4ee4-a8d1-d440506c3784 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:21.947174 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-7ce0452c-dfa6-4ee4-a8d1-d440506c3784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:21.956751 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1434/2861] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:21.961156 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18f1d0-653c-47c7-befa-ff2ffff31571 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:21.971361 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18f1d0-653c-47c7-befa-ff2ffff31571 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:21.976590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd603da-5a93-488e-a58c-3c8a28ad7488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:21.977249 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd603da-5a93-488e-a58c-3c8a28ad7488 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:21.977512 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd603da-5a93-488e-a58c-3c8a28ad7488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:21.977806 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd603da-5a93-488e-a58c-3c8a28ad7488 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:21.978535 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd603da-5a93-488e-a58c-3c8a28ad7488 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:09:21.978842 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd603da-5a93-488e-a58c-3c8a28ad7488 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:21.980552 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd603da-5a93-488e-a58c-3c8a28ad7488 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:21.992136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1428/2862] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 06:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:09:21.996389 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0452c-dfa6-4ee4-a8d1-d440506c3784 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:22.005717 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0452c-dfa6-4ee4-a8d1-d440506c3784 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:22.010780 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ebe20-6cb5-4f02-9e2e-9b0283b80fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:22.011486 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ebe20-6cb5-4f02-9e2e-9b0283b80fbc admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:22.011769 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ebe20-6cb5-4f02-9e2e-9b0283b80fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:22.012045 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ebe20-6cb5-4f02-9e2e-9b0283b80fbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:22.012747 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ebe20-6cb5-4f02-9e2e-9b0283b80fbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:22.014692 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ebe20-6cb5-4f02-9e2e-9b0283b80fbc admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:22.044349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1435/2863] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 06:09:21 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:09:22.048598 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd603da-5a93-488e-a58c-3c8a28ad7488 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:22.058666 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd603da-5a93-488e-a58c-3c8a28ad7488 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:22.084881 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:22.124636 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa10bc-da49-454c-8f99-feed22c334b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:22.125289 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa10bc-da49-454c-8f99-feed22c334b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:22.125569 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa10bc-da49-454c-8f99-feed22c334b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:22.125841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa10bc-da49-454c-8f99-feed22c334b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:22.126524 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa10bc-da49-454c-8f99-feed22c334b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:22.128475 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa10bc-da49-454c-8f99-feed22c334b8 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:22.147997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1429/2864] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 06:09:22 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:09:22.152232 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ebe20-6cb5-4f02-9e2e-9b0283b80fbc admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:22.163346 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ebe20-6cb5-4f02-9e2e-9b0283b80fbc admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:22.169046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7339ed6-c396-4bcc-940c-1b5d3a5f0759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:22.169710 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7339ed6-c396-4bcc-940c-1b5d3a5f0759 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:22.170028 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7339ed6-c396-4bcc-940c-1b5d3a5f0759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:22.170299 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7339ed6-c396-4bcc-940c-1b5d3a5f0759 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:22.171047 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7339ed6-c396-4bcc-940c-1b5d3a5f0759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:22.172738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7339ed6-c396-4bcc-940c-1b5d3a5f0759 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:22.183852 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1436/2865] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 06:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:09:22.188262 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa10bc-da49-454c-8f99-feed22c334b8 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:22.199246 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa10bc-da49-454c-8f99-feed22c334b8 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:22.204905 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae53a3a-d3b7-4193-866c-db003fa1285c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:22.205572 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae53a3a-d3b7-4193-866c-db003fa1285c admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:22.205860 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae53a3a-d3b7-4193-866c-db003fa1285c admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:22.206136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae53a3a-d3b7-4193-866c-db003fa1285c admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:22.206870 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae53a3a-d3b7-4193-866c-db003fa1285c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:22.208501 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae53a3a-d3b7-4193-866c-db003fa1285c admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:22.221487 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1430/2866] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 06:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:09:22.226309 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7339ed6-c396-4bcc-940c-1b5d3a5f0759 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:22.238055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7339ed6-c396-4bcc-940c-1b5d3a5f0759 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:22.243674 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cf3ac-0dcd-4532-b1e9-d7c2c81d84d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:22.244565 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cf3ac-0dcd-4532-b1e9-d7c2c81d84d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:22.244849 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cf3ac-0dcd-4532-b1e9-d7c2c81d84d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:22.245151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cf3ac-0dcd-4532-b1e9-d7c2c81d84d7 admin admin] PATH_INFO: `/v3/projects/ca0f297a292e45e5bf21ed5dc98c3e99/users/73a8cf4040994e16804b37c20afd2789/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:22.260909 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5cf3ac-0dcd-4532-b1e9-d7c2c81d84d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca0f297a292e45e5bf21ed5dc98c3e99, user_id=73a8cf4040994e16804b37c20afd2789, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:22.263731 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5cf3ac-0dcd-4532-b1e9-d7c2c81d84d7 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:22.278807 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1437/2867] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 06:09:22 2022] PUT /identity/v3/projects/ca0f297a292e45e5bf21ed5dc98c3e99/users/73a8cf4040994e16804b37c20afd2789/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:09:22.290583 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fd5c5-1994-455a-b1f5-5dfd7d82c5f9 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:22.290872 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fd5c5-1994-455a-b1f5-5dfd7d82c5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:22.291186 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fd5c5-1994-455a-b1f5-5dfd7d82c5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:22.318175 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-6e1fd5c5-1994-455a-b1f5-5dfd7d82c5f9 None None] MFA Rules not processed for user `73a8cf4040994e16804b37c20afd2789`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:22.319410 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-6e1fd5c5-1994-455a-b1f5-5dfd7d82c5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:22.382217 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1431/2868] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:09:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:22.387030 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cf3ac-0dcd-4532-b1e9-d7c2c81d84d7 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:22.458205 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cf3ac-0dcd-4532-b1e9-d7c2c81d84d7 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:22.467731 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e61d7d-d288-4974-9c3b-1397c9612625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:22.468768 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e61d7d-d288-4974-9c3b-1397c9612625 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:22.469243 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e61d7d-d288-4974-9c3b-1397c9612625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:22.469620 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e61d7d-d288-4974-9c3b-1397c9612625 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:22.470758 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e61d7d-d288-4974-9c3b-1397c9612625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:22.473978 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e61d7d-d288-4974-9c3b-1397c9612625 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:22.508398 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1438/2869] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 06:09:22 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:09:22.513801 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1fd5c5-1994-455a-b1f5-5dfd7d82c5f9 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:22.525169 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1fd5c5-1994-455a-b1f5-5dfd7d82c5f9 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:22.592812 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f5524-536e-465f-bc10-c109fad17594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:22.593637 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f5524-536e-465f-bc10-c109fad17594 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:22.593994 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f5524-536e-465f-bc10-c109fad17594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:22.594292 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f5524-536e-465f-bc10-c109fad17594 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:22.595369 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f5524-536e-465f-bc10-c109fad17594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:22.597749 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f5524-536e-465f-bc10-c109fad17594 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:22.623264 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1432/2870] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 06:09:22 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:09:22.627199 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e61d7d-d288-4974-9c3b-1397c9612625 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:22.638333 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e61d7d-d288-4974-9c3b-1397c9612625 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:22.643460 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58428829-fc43-4a49-838f-3d3eeec2f8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:22.644818 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58428829-fc43-4a49-838f-3d3eeec2f8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:22.645100 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58428829-fc43-4a49-838f-3d3eeec2f8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:22.645380 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58428829-fc43-4a49-838f-3d3eeec2f8f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:22.646117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58428829-fc43-4a49-838f-3d3eeec2f8f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:22.647767 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58428829-fc43-4a49-838f-3d3eeec2f8f6 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:22.661239 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1439/2871] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 06:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:09:22.663980 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f5524-536e-465f-bc10-c109fad17594 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:22.674452 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f5524-536e-465f-bc10-c109fad17594 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:22.679307 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5aabc-75fd-4016-8743-19fe37a34ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:22.680192 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5aabc-75fd-4016-8743-19fe37a34ce5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:22.680475 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5aabc-75fd-4016-8743-19fe37a34ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:22.680752 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5aabc-75fd-4016-8743-19fe37a34ce5 admin admin] PATH_INFO: `/v3/projects/7b51a1f7553c482bab13a38fd306d3b3/users/07072a1246c2426f800fb32dab5a6024/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:22.695457 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5aabc-75fd-4016-8743-19fe37a34ce5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b51a1f7553c482bab13a38fd306d3b3, user_id=07072a1246c2426f800fb32dab5a6024, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:22.698248 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5aabc-75fd-4016-8743-19fe37a34ce5 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:22.711907 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1433/2872] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 06:09:22 2022] PUT /identity/v3/projects/7b51a1f7553c482bab13a38fd306d3b3/users/07072a1246c2426f800fb32dab5a6024/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:09:22.716050 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58428829-fc43-4a49-838f-3d3eeec2f8f6 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:22.726879 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58428829-fc43-4a49-838f-3d3eeec2f8f6 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:22.747941 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:22.786943 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aaecb6-94f0-4408-a760-923e10bd896c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:22.787651 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aaecb6-94f0-4408-a760-923e10bd896c admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:22.787934 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aaecb6-94f0-4408-a760-923e10bd896c admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:22.788215 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aaecb6-94f0-4408-a760-923e10bd896c admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:22.788938 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aaecb6-94f0-4408-a760-923e10bd896c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:22.790534 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aaecb6-94f0-4408-a760-923e10bd896c admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:22.800554 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1440/2873] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 06:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:09:22.805291 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5aabc-75fd-4016-8743-19fe37a34ce5 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:22.811468 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:22.819529 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5aabc-75fd-4016-8743-19fe37a34ce5 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:22.827434 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4733c7-4d38-418e-b472-747076a834cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:22.828954 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4733c7-4d38-418e-b472-747076a834cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:22.829470 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4733c7-4d38-418e-b472-747076a834cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:22.829933 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4733c7-4d38-418e-b472-747076a834cf admin admin] PATH_INFO: `/v3/projects/7b51a1f7553c482bab13a38fd306d3b3/users/07072a1246c2426f800fb32dab5a6024/roles/6a5988fad9e245eba3c55eba854db4e8` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:22.833841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4733c7-4d38-418e-b472-747076a834cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b51a1f7553c482bab13a38fd306d3b3, user_id=07072a1246c2426f800fb32dab5a6024, role_id=6a5988fad9e245eba3c55eba854db4e8)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:22.838259 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4733c7-4d38-418e-b472-747076a834cf admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:22.854544 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1434/2874] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 06:09:22 2022] PUT /identity/v3/projects/7b51a1f7553c482bab13a38fd306d3b3/users/07072a1246c2426f800fb32dab5a6024/roles/6a5988fad9e245eba3c55eba854db4e8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 21 06:09:22.865138 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a002b-97eb-435d-8ce5-2fe97a81a931 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:22.865428 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a002b-97eb-435d-8ce5-2fe97a81a931 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:22.865706 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a002b-97eb-435d-8ce5-2fe97a81a931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:22.893474 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-a21a002b-97eb-435d-8ce5-2fe97a81a931 None None] MFA Rules not processed for user `07072a1246c2426f800fb32dab5a6024`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:22.894693 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a21a002b-97eb-435d-8ce5-2fe97a81a931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:22.967642 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1441/2875] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:09:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:22.978811 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4733c7-4d38-418e-b472-747076a834cf admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:22.980997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-7c4733c7-4d38-418e-b472-747076a834cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:23.042584 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4733c7-4d38-418e-b472-747076a834cf admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:23.049687 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd12d31-0019-41b6-b47b-b3b5a229a5b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:23.050574 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd12d31-0019-41b6-b47b-b3b5a229a5b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:23.050864 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd12d31-0019-41b6-b47b-b3b5a229a5b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:23.051193 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd12d31-0019-41b6-b47b-b3b5a229a5b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:23.062630 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd12d31-0019-41b6-b47b-b3b5a229a5b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:23.064126 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd12d31-0019-41b6-b47b-b3b5a229a5b4 service cinder] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:23.074158 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1435/2876] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:09:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:09:23.179804 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a002b-97eb-435d-8ce5-2fe97a81a931 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:23.190361 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a002b-97eb-435d-8ce5-2fe97a81a931 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:23.194805 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4158c-119c-49d1-bf1b-b501fce146cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:23.195478 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4158c-119c-49d1-bf1b-b501fce146cf service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:23.195781 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4158c-119c-49d1-bf1b-b501fce146cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:23.196054 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4158c-119c-49d1-bf1b-b501fce146cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:23.256013 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4158c-119c-49d1-bf1b-b501fce146cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:23.257470 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc4158c-119c-49d1-bf1b-b501fce146cf service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:23.270655 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1442/2877] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:09:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:09:25.904756 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd12d31-0019-41b6-b47b-b3b5a229a5b4 service cinder] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:25.969348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd12d31-0019-41b6-b47b-b3b5a229a5b4 service cinder] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:25.974418 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5fa4c-1d4d-4d33-8b15-932549af2c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:25.975322 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5fa4c-1d4d-4d33-8b15-932549af2c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:25.975599 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5fa4c-1d4d-4d33-8b15-932549af2c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:25.975870 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5fa4c-1d4d-4d33-8b15-932549af2c4c admin admin] PATH_INFO: `/v3/users/73a8cf4040994e16804b37c20afd2789` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:25.977091 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5fa4c-1d4d-4d33-8b15-932549af2c4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a8cf4040994e16804b37c20afd2789)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:25.979037 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5fa4c-1d4d-4d33-8b15-932549af2c4c admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:26.033118 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-baa5fa4c-1d4d-4d33-8b15-932549af2c4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a8cf4040994e16804b37c20afd2789'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:26.045883 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-baa5fa4c-1d4d-4d33-8b15-932549af2c4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a8cf4040994e16804b37c20afd2789'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:26.052896 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-baa5fa4c-1d4d-4d33-8b15-932549af2c4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a8cf4040994e16804b37c20afd2789'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:26.053656 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-baa5fa4c-1d4d-4d33-8b15-932549af2c4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a8cf4040994e16804b37c20afd2789'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:26.054383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-baa5fa4c-1d4d-4d33-8b15-932549af2c4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a8cf4040994e16804b37c20afd2789'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:26.069611 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1436/2878] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 06:09:25 2022] DELETE /identity/v3/users/73a8cf4040994e16804b37c20afd2789 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:09:26.074348 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f36f5-66c6-4f37-b51b-e37554b3564c None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:26.074823 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f36f5-66c6-4f37-b51b-e37554b3564c None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:26.074918 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f36f5-66c6-4f37-b51b-e37554b3564c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:26.092355 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-276f36f5-66c6-4f37-b51b-e37554b3564c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:26.093558 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-276f36f5-66c6-4f37-b51b-e37554b3564c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:26.134224 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=100 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:26.154396 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1443/2879] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:09:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:26.164863 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5fa4c-1d4d-4d33-8b15-932549af2c4c admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:26.166345 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-baa5fa4c-1d4d-4d33-8b15-932549af2c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:26.180244 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:26.237924 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5fa4c-1d4d-4d33-8b15-932549af2c4c admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:26.241940 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6aa52-b529-40a4-86a9-48d0c8d87035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:26.242590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6aa52-b529-40a4-86a9-48d0c8d87035 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:26.242877 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6aa52-b529-40a4-86a9-48d0c8d87035 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:26.243183 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6aa52-b529-40a4-86a9-48d0c8d87035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:26.253186 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6aa52-b529-40a4-86a9-48d0c8d87035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:26.254627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6aa52-b529-40a4-86a9-48d0c8d87035 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:26.264746 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1437/2880] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:09:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:09:26.546418 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f36f5-66c6-4f37-b51b-e37554b3564c None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:26.548021 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-276f36f5-66c6-4f37-b51b-e37554b3564c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:26.562699 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f36f5-66c6-4f37-b51b-e37554b3564c None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:26.568662 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:26.569692 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:26.569831 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:26.570111 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4 admin admin] PATH_INFO: `/v3/projects/ca0f297a292e45e5bf21ed5dc98c3e99` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:26.571525 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca0f297a292e45e5bf21ed5dc98c3e99)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:26.573511 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:26.622190 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca0f297a292e45e5bf21ed5dc98c3e99', 'request_id': 'req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:26.627892 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca0f297a292e45e5bf21ed5dc98c3e99', 'request_id': 'req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:26.636249 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca0f297a292e45e5bf21ed5dc98c3e99', 'request_id': 'req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:26.647221 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4 admin admin] The token cache is being invalidate because project ca0f297a292e45e5bf21ed5dc98c3e99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:09:26.647494 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:26.648128 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:26.649666 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1444/2881] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 06:09:26 2022] DELETE /identity/v3/projects/ca0f297a292e45e5bf21ed5dc98c3e99 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:09:26.653866 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6aa52-b529-40a4-86a9-48d0c8d87035 service neutron] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:26.664547 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6aa52-b529-40a4-86a9-48d0c8d87035 service neutron] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:26.727573 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42987cd4-c77a-4d1d-aca3-cba7d6b15154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:26.728452 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42987cd4-c77a-4d1d-aca3-cba7d6b15154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:26.728730 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42987cd4-c77a-4d1d-aca3-cba7d6b15154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:26.729003 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42987cd4-c77a-4d1d-aca3-cba7d6b15154 admin admin] PATH_INFO: `/v3/users/07072a1246c2426f800fb32dab5a6024` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:26.730151 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42987cd4-c77a-4d1d-aca3-cba7d6b15154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07072a1246c2426f800fb32dab5a6024)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:26.732300 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42987cd4-c77a-4d1d-aca3-cba7d6b15154 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:26.781274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-42987cd4-c77a-4d1d-aca3-cba7d6b15154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07072a1246c2426f800fb32dab5a6024'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:26.792494 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-42987cd4-c77a-4d1d-aca3-cba7d6b15154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07072a1246c2426f800fb32dab5a6024'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:26.799296 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-42987cd4-c77a-4d1d-aca3-cba7d6b15154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07072a1246c2426f800fb32dab5a6024'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:26.799999 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-42987cd4-c77a-4d1d-aca3-cba7d6b15154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07072a1246c2426f800fb32dab5a6024'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:26.800653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-42987cd4-c77a-4d1d-aca3-cba7d6b15154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07072a1246c2426f800fb32dab5a6024'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:26.814735 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1438/2882] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 06:09:26 2022] DELETE /identity/v3/users/07072a1246c2426f800fb32dab5a6024 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:09:27.104220 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:27.105753 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:27.120040 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6e811-3951-4d6a-9e13-703ee4ce9ce4 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:27.179156 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e3e5d-159c-4cd7-a579-4436fb93d011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:27.180033 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e3e5d-159c-4cd7-a579-4436fb93d011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:27.180315 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e3e5d-159c-4cd7-a579-4436fb93d011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:27.180589 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e3e5d-159c-4cd7-a579-4436fb93d011 admin admin] PATH_INFO: `/v3/projects/7b51a1f7553c482bab13a38fd306d3b3` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:27.181766 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e3e5d-159c-4cd7-a579-4436fb93d011 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b51a1f7553c482bab13a38fd306d3b3)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:27.183732 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e3e5d-159c-4cd7-a579-4436fb93d011 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:27.233539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-251e3e5d-159c-4cd7-a579-4436fb93d011 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b51a1f7553c482bab13a38fd306d3b3', 'request_id': 'req-251e3e5d-159c-4cd7-a579-4436fb93d011'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:27.239265 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-251e3e5d-159c-4cd7-a579-4436fb93d011 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b51a1f7553c482bab13a38fd306d3b3', 'request_id': 'req-251e3e5d-159c-4cd7-a579-4436fb93d011'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:27.247751 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-251e3e5d-159c-4cd7-a579-4436fb93d011 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b51a1f7553c482bab13a38fd306d3b3', 'request_id': 'req-251e3e5d-159c-4cd7-a579-4436fb93d011'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:27.255199 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:27.261136 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-251e3e5d-159c-4cd7-a579-4436fb93d011 admin admin] The token cache is being invalidate because project 7b51a1f7553c482bab13a38fd306d3b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:09:27.261429 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-251e3e5d-159c-4cd7-a579-4436fb93d011 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:27.262422 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-251e3e5d-159c-4cd7-a579-4436fb93d011 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:27.264233 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1445/2883] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 06:09:27 2022] DELETE /identity/v3/projects/7b51a1f7553c482bab13a38fd306d3b3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:09:27.270014 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea74441-276e-43e1-a5b9-1baf6d24a66a None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:27.270532 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea74441-276e-43e1-a5b9-1baf6d24a66a None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:27.270653 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea74441-276e-43e1-a5b9-1baf6d24a66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:27.289467 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-dea74441-276e-43e1-a5b9-1baf6d24a66a None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:27.290692 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-dea74441-276e-43e1-a5b9-1baf6d24a66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:27.321757 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:27.352647 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1439/2884] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:09:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:27.360817 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a986aaa-6b3f-45e9-9a6c-c053b60311f9 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:27.361109 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a986aaa-6b3f-45e9-9a6c-c053b60311f9 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:27.361415 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a986aaa-6b3f-45e9-9a6c-c053b60311f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:27.380849 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-8a986aaa-6b3f-45e9-9a6c-c053b60311f9 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:27.382046 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8a986aaa-6b3f-45e9-9a6c-c053b60311f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:27.390567 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1446/2885] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:09:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:27.394945 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea74441-276e-43e1-a5b9-1baf6d24a66a None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:27.405352 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea74441-276e-43e1-a5b9-1baf6d24a66a None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:27.410256 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460a705-64a9-42f9-a5ff-5dc87e02fc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:27.410918 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460a705-64a9-42f9-a5ff-5dc87e02fc08 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:27.411321 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460a705-64a9-42f9-a5ff-5dc87e02fc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:27.411602 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460a705-64a9-42f9-a5ff-5dc87e02fc08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:27.412330 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460a705-64a9-42f9-a5ff-5dc87e02fc08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:09:27.412636 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460a705-64a9-42f9-a5ff-5dc87e02fc08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:27.414250 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460a705-64a9-42f9-a5ff-5dc87e02fc08 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:27.425907 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1440/2886] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 06:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:09:27.430225 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a986aaa-6b3f-45e9-9a6c-c053b60311f9 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:27.440035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a986aaa-6b3f-45e9-9a6c-c053b60311f9 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:27.445318 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482a0af-aeb8-4e28-b760-ba846cd36b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:27.445959 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482a0af-aeb8-4e28-b760-ba846cd36b55 admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:27.446240 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482a0af-aeb8-4e28-b760-ba846cd36b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:27.446526 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482a0af-aeb8-4e28-b760-ba846cd36b55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:27.447249 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482a0af-aeb8-4e28-b760-ba846cd36b55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:27.449172 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482a0af-aeb8-4e28-b760-ba846cd36b55 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:27.477245 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1447/2887] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 06:09:27 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:09:27.481389 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460a705-64a9-42f9-a5ff-5dc87e02fc08 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:27.491315 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460a705-64a9-42f9-a5ff-5dc87e02fc08 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:27.548758 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1f777-d957-480d-a637-3ed72bb7c91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:27.549435 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1f777-d957-480d-a637-3ed72bb7c91b admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:27.549718 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1f777-d957-480d-a637-3ed72bb7c91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:27.549995 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1f777-d957-480d-a637-3ed72bb7c91b admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:27.550684 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1f777-d957-480d-a637-3ed72bb7c91b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:27.552692 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1f777-d957-480d-a637-3ed72bb7c91b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:27.574890 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1441/2888] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 06:09:27 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:09:27.579197 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482a0af-aeb8-4e28-b760-ba846cd36b55 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:27.589757 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482a0af-aeb8-4e28-b760-ba846cd36b55 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:27.595868 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d515f8b-f314-4ba6-ac62-f78a2ac482a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:27.596538 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d515f8b-f314-4ba6-ac62-f78a2ac482a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:27.596827 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d515f8b-f314-4ba6-ac62-f78a2ac482a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:27.597118 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d515f8b-f314-4ba6-ac62-f78a2ac482a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:27.597863 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d515f8b-f314-4ba6-ac62-f78a2ac482a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:27.599523 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d515f8b-f314-4ba6-ac62-f78a2ac482a7 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:27.610537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1448/2889] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 06:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:09:27.614488 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1f777-d957-480d-a637-3ed72bb7c91b admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:27.624665 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1f777-d957-480d-a637-3ed72bb7c91b admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:27.629635 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15412932-7601-4cac-9167-1cb93b99916b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:27.630349 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15412932-7601-4cac-9167-1cb93b99916b admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:27.630690 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15412932-7601-4cac-9167-1cb93b99916b admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:27.631000 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15412932-7601-4cac-9167-1cb93b99916b admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:27.631757 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15412932-7601-4cac-9167-1cb93b99916b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:27.633383 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15412932-7601-4cac-9167-1cb93b99916b admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:27.643791 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1442/2890] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 06:09:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:09:27.648383 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d515f8b-f314-4ba6-ac62-f78a2ac482a7 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:27.659417 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d515f8b-f314-4ba6-ac62-f78a2ac482a7 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:27.664461 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad218a-3039-4797-be25-570eedc4650e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:27.665393 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad218a-3039-4797-be25-570eedc4650e admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:27.665650 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad218a-3039-4797-be25-570eedc4650e admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:27.665926 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad218a-3039-4797-be25-570eedc4650e admin admin] PATH_INFO: `/v3/projects/d238f256864543dfae6ae044b7e0746f/users/522f305d299a4500a6e1ad89a31ade4e/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:27.682691 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad218a-3039-4797-be25-570eedc4650e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d238f256864543dfae6ae044b7e0746f, user_id=522f305d299a4500a6e1ad89a31ade4e, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:27.685535 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad218a-3039-4797-be25-570eedc4650e admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:27.702673 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1449/2891] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 06:09:27 2022] PUT /identity/v3/projects/d238f256864543dfae6ae044b7e0746f/users/522f305d299a4500a6e1ad89a31ade4e/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:09:27.714997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a46562-4231-4d5c-9585-948921f4cdfd None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:27.715319 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a46562-4231-4d5c-9585-948921f4cdfd None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:27.715602 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a46562-4231-4d5c-9585-948921f4cdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:27.743715 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-a3a46562-4231-4d5c-9585-948921f4cdfd None None] MFA Rules not processed for user `522f305d299a4500a6e1ad89a31ade4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:27.744927 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-a3a46562-4231-4d5c-9585-948921f4cdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:27.812612 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1443/2892] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:09:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:27.823807 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad218a-3039-4797-be25-570eedc4650e admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:27.825955 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-c7ad218a-3039-4797-be25-570eedc4650e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:27.894738 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad218a-3039-4797-be25-570eedc4650e admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:27.899129 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b43585-c9f6-4326-8845-8bc5d0accc08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:27.899790 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b43585-c9f6-4326-8845-8bc5d0accc08 service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:27.900075 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b43585-c9f6-4326-8845-8bc5d0accc08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:27.900353 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b43585-c9f6-4326-8845-8bc5d0accc08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:27.909539 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b43585-c9f6-4326-8845-8bc5d0accc08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:27.910986 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b43585-c9f6-4326-8845-8bc5d0accc08 service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:27.920105 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1450/2893] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:09:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:09:36.573314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a46562-4231-4d5c-9585-948921f4cdfd None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:36.605627 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:36.637361 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a46562-4231-4d5c-9585-948921f4cdfd None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:36.642379 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a243d0-bd9d-4667-9acd-f779c466556d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:36.643286 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a243d0-bd9d-4667-9acd-f779c466556d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:36.643565 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a243d0-bd9d-4667-9acd-f779c466556d admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:36.643841 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a243d0-bd9d-4667-9acd-f779c466556d admin admin] PATH_INFO: `/v3/users/522f305d299a4500a6e1ad89a31ade4e` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:36.645051 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a243d0-bd9d-4667-9acd-f779c466556d admin admin] RBAC: Authorizing `identity:delete_user(user_id=522f305d299a4500a6e1ad89a31ade4e)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:36.646990 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a243d0-bd9d-4667-9acd-f779c466556d admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:36.698579 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-29a243d0-bd9d-4667-9acd-f779c466556d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '522f305d299a4500a6e1ad89a31ade4e'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:36.710508 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-29a243d0-bd9d-4667-9acd-f779c466556d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '522f305d299a4500a6e1ad89a31ade4e'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:36.717153 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-29a243d0-bd9d-4667-9acd-f779c466556d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '522f305d299a4500a6e1ad89a31ade4e'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:36.717896 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-29a243d0-bd9d-4667-9acd-f779c466556d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '522f305d299a4500a6e1ad89a31ade4e'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:36.718561 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-29a243d0-bd9d-4667-9acd-f779c466556d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '522f305d299a4500a6e1ad89a31ade4e'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:36.733732 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1444/2894] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 06:09:36 2022] DELETE /identity/v3/users/522f305d299a4500a6e1ad89a31ade4e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:09:36.738435 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70c64a-c94f-4cf6-80a3-88cf32466d71 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:36.738974 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70c64a-c94f-4cf6-80a3-88cf32466d71 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:36.738974 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70c64a-c94f-4cf6-80a3-88cf32466d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:36.756341 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-8c70c64a-c94f-4cf6-80a3-88cf32466d71 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:36.757542 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8c70c64a-c94f-4cf6-80a3-88cf32466d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:36.823571 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1451/2895] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:36.833695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a243d0-bd9d-4667-9acd-f779c466556d admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:36.835160 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-29a243d0-bd9d-4667-9acd-f779c466556d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:36.892492 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a243d0-bd9d-4667-9acd-f779c466556d admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:36.896618 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815c215-51f3-4518-8794-bf47f2f90e97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:36.897261 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815c215-51f3-4518-8794-bf47f2f90e97 service neutron] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:36.897553 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815c215-51f3-4518-8794-bf47f2f90e97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:36.897829 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815c215-51f3-4518-8794-bf47f2f90e97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:36.907732 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815c215-51f3-4518-8794-bf47f2f90e97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:36.909191 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815c215-51f3-4518-8794-bf47f2f90e97 service neutron] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:36.918844 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1445/2896] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:09:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:09:37.210314 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70c64a-c94f-4cf6-80a3-88cf32466d71 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:37.211969 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8c70c64a-c94f-4cf6-80a3-88cf32466d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:37.227950 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70c64a-c94f-4cf6-80a3-88cf32466d71 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:37.233605 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237212d-251b-4a08-ae01-544923738ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:37.234452 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237212d-251b-4a08-ae01-544923738ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:37.234754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237212d-251b-4a08-ae01-544923738ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:37.235034 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237212d-251b-4a08-ae01-544923738ace admin admin] PATH_INFO: `/v3/projects/d238f256864543dfae6ae044b7e0746f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:37.238535 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237212d-251b-4a08-ae01-544923738ace admin admin] RBAC: Authorizing `identity:delete_project(project_id=d238f256864543dfae6ae044b7e0746f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:37.238750 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237212d-251b-4a08-ae01-544923738ace admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:37.274473 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:37.288246 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b237212d-251b-4a08-ae01-544923738ace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd238f256864543dfae6ae044b7e0746f', 'request_id': 'req-b237212d-251b-4a08-ae01-544923738ace'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:37.293602 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b237212d-251b-4a08-ae01-544923738ace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd238f256864543dfae6ae044b7e0746f', 'request_id': 'req-b237212d-251b-4a08-ae01-544923738ace'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:37.301592 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b237212d-251b-4a08-ae01-544923738ace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd238f256864543dfae6ae044b7e0746f', 'request_id': 'req-b237212d-251b-4a08-ae01-544923738ace'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:37.313141 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b237212d-251b-4a08-ae01-544923738ace admin admin] The token cache is being invalidate because project d238f256864543dfae6ae044b7e0746f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:09:37.313425 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b237212d-251b-4a08-ae01-544923738ace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:37.314052 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-b237212d-251b-4a08-ae01-544923738ace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:37.315644 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1452/2897] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 06:09:37 2022] DELETE /identity/v3/projects/d238f256864543dfae6ae044b7e0746f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:09:37.320714 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57bf7e-c962-4c06-b0f1-3d51da0b4cb5 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:37.321035 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57bf7e-c962-4c06-b0f1-3d51da0b4cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:37.321314 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57bf7e-c962-4c06-b0f1-3d51da0b4cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:37.338868 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-4d57bf7e-c962-4c06-b0f1-3d51da0b4cb5 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:37.340085 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-4d57bf7e-c962-4c06-b0f1-3d51da0b4cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:37.386787 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:37.402461 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1446/2898] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:37.411105 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9a454-51dd-4338-8706-10b88b3cebd0 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:37.411484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9a454-51dd-4338-8706-10b88b3cebd0 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:37.411796 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9a454-51dd-4338-8706-10b88b3cebd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:37.431195 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-0ed9a454-51dd-4338-8706-10b88b3cebd0 None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:37.432394 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-0ed9a454-51dd-4338-8706-10b88b3cebd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:37.441578 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1453/2899] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:37.445850 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d57bf7e-c962-4c06-b0f1-3d51da0b4cb5 None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:37.456452 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d57bf7e-c962-4c06-b0f1-3d51da0b4cb5 None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:37.461776 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc985d-e4a9-4982-8473-5c6986f12902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:37.462417 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc985d-e4a9-4982-8473-5c6986f12902 admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:37.462792 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc985d-e4a9-4982-8473-5c6986f12902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:37.463100 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc985d-e4a9-4982-8473-5c6986f12902 admin admin] PATH_INFO: `/v3/domains` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:37.463844 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc985d-e4a9-4982-8473-5c6986f12902 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80391) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 21 06:09:37.464157 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc985d-e4a9-4982-8473-5c6986f12902 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:37.465769 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc985d-e4a9-4982-8473-5c6986f12902 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:37.478487 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1447/2900] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jul 21 06:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 21 06:09:37.483190 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9a454-51dd-4338-8706-10b88b3cebd0 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:37.492791 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9a454-51dd-4338-8706-10b88b3cebd0 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:37.498896 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b89892-1ff4-4352-bd2d-73134c04514a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:37.499595 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b89892-1ff4-4352-bd2d-73134c04514a admin admin] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:37.499877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b89892-1ff4-4352-bd2d-73134c04514a admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:37.500149 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b89892-1ff4-4352-bd2d-73134c04514a admin admin] PATH_INFO: `/v3/projects` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:37.500844 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b89892-1ff4-4352-bd2d-73134c04514a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:37.502793 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b89892-1ff4-4352-bd2d-73134c04514a admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:37.531753 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1454/2901] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jul 21 06:09:37 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 21 06:09:37.536449 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc985d-e4a9-4982-8473-5c6986f12902 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:37.549928 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc985d-e4a9-4982-8473-5c6986f12902 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:37.612578 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f8687-acc0-4dfd-ba40-7966f31dde68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:37.613290 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f8687-acc0-4dfd-ba40-7966f31dde68 admin admin] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:37.613590 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f8687-acc0-4dfd-ba40-7966f31dde68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:37.613856 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f8687-acc0-4dfd-ba40-7966f31dde68 admin admin] PATH_INFO: `/v3/users` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:37.614578 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f8687-acc0-4dfd-ba40-7966f31dde68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:37.616545 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f8687-acc0-4dfd-ba40-7966f31dde68 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:37.638309 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1448/2902] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jul 21 06:09:37 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 21 06:09:37.642586 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b89892-1ff4-4352-bd2d-73134c04514a admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:37.652967 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b89892-1ff4-4352-bd2d-73134c04514a admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:37.658208 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d91404-7985-402e-9a48-7a55a842d7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:37.658877 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d91404-7985-402e-9a48-7a55a842d7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:37.659193 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d91404-7985-402e-9a48-7a55a842d7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:37.659470 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d91404-7985-402e-9a48-7a55a842d7a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:37.660201 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d91404-7985-402e-9a48-7a55a842d7a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:37.661804 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d91404-7985-402e-9a48-7a55a842d7a0 admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:37.672821 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1455/2903] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 06:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:09:37.677002 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f8687-acc0-4dfd-ba40-7966f31dde68 admin admin] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:37.687430 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f8687-acc0-4dfd-ba40-7966f31dde68 admin admin] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:37.692538 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9fe713-a34e-4d34-8a0d-6d487755d61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:37.693197 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9fe713-a34e-4d34-8a0d-6d487755d61c admin admin] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:37.693481 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9fe713-a34e-4d34-8a0d-6d487755d61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:37.693788 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9fe713-a34e-4d34-8a0d-6d487755d61c admin admin] PATH_INFO: `/v3/roles` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:37.694512 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9fe713-a34e-4d34-8a0d-6d487755d61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:37.696180 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9fe713-a34e-4d34-8a0d-6d487755d61c admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:37.706243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1449/2904] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jul 21 06:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 21 06:09:37.710446 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d91404-7985-402e-9a48-7a55a842d7a0 admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:37.720718 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d91404-7985-402e-9a48-7a55a842d7a0 admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:37.726280 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4b827-51dc-4cd8-a812-a0ac202a893b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:37.727198 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4b827-51dc-4cd8-a812-a0ac202a893b admin admin] REQUEST_METHOD: `PUT` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:37.727484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4b827-51dc-4cd8-a812-a0ac202a893b admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:37.727756 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4b827-51dc-4cd8-a812-a0ac202a893b admin admin] PATH_INFO: `/v3/projects/412742fe621f4e63acb5891ae57a10d0/users/3d7bbfba930843cd9782c4bf4931c3b1/roles/b59c5b350c854281838a22884fd8f32f` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:37.746894 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4b827-51dc-4cd8-a812-a0ac202a893b admin admin] RBAC: Authorizing `identity:create_grant(project_id=412742fe621f4e63acb5891ae57a10d0, user_id=3d7bbfba930843cd9782c4bf4931c3b1, role_id=b59c5b350c854281838a22884fd8f32f)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:37.750799 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4b827-51dc-4cd8-a812-a0ac202a893b admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:37.769438 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1456/2905] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jul 21 06:09:37 2022] PUT /identity/v3/projects/412742fe621f4e63acb5891ae57a10d0/users/3d7bbfba930843cd9782c4bf4931c3b1/roles/b59c5b350c854281838a22884fd8f32f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:09:37.783932 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa94665-932d-439e-b30b-01d275f098c4 None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:37.784221 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa94665-932d-439e-b30b-01d275f098c4 None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:37.784495 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa94665-932d-439e-b30b-01d275f098c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:37.812490 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-1fa94665-932d-439e-b30b-01d275f098c4 None None] MFA Rules not processed for user `3d7bbfba930843cd9782c4bf4931c3b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:37.813682 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-1fa94665-932d-439e-b30b-01d275f098c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:37.880243 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1450/2906] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:09:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:37.894035 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4b827-51dc-4cd8-a812-a0ac202a893b admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:37.895855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-7ff4b827-51dc-4cd8-a812-a0ac202a893b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:37.953994 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4b827-51dc-4cd8-a812-a0ac202a893b admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:37.958049 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372429c8-6d73-4eb9-af0c-aef7f8e8603a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3f76371c6a04fc5a477d6ef1908a08e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:37.958690 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372429c8-6d73-4eb9-af0c-aef7f8e8603a service cinder] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:37.958975 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372429c8-6d73-4eb9-af0c-aef7f8e8603a service cinder] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:37.959293 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372429c8-6d73-4eb9-af0c-aef7f8e8603a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:37.968213 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372429c8-6d73-4eb9-af0c-aef7f8e8603a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:37.969596 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372429c8-6d73-4eb9-af0c-aef7f8e8603a service cinder] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:37.978664 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1457/2907] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:09:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:09:39.344719 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9326850-bc68-47a2-9116-4cf8ae5669cd None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:39.345278 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9326850-bc68-47a2-9116-4cf8ae5669cd None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:39.345278 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9326850-bc68-47a2-9116-4cf8ae5669cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:39.365019 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-c9326850-bc68-47a2-9116-4cf8ae5669cd None None] MFA Rules not processed for user `3d7bbfba930843cd9782c4bf4931c3b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:39.366214 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-c9326850-bc68-47a2-9116-4cf8ae5669cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:39.374380 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1451/2908] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:09:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:39.849537 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372429c8-6d73-4eb9-af0c-aef7f8e8603a service cinder] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:39.851077 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-372429c8-6d73-4eb9-af0c-aef7f8e8603a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:39.918890 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:39.925359 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372429c8-6d73-4eb9-af0c-aef7f8e8603a service cinder] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:39.931530 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebcaac-8419-4d8d-a86d-0e57210f73a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:39.932379 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebcaac-8419-4d8d-a86d-0e57210f73a2 service placement] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:39.932813 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebcaac-8419-4d8d-a86d-0e57210f73a2 service placement] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:39.933303 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebcaac-8419-4d8d-a86d-0e57210f73a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:39.935117 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-b3ebcaac-8419-4d8d-a86d-0e57210f73a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:39.946645 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebcaac-8419-4d8d-a86d-0e57210f73a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:39.948272 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebcaac-8419-4d8d-a86d-0e57210f73a2 service placement] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:39.958613 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1458/2909] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:09:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 21 06:09:49.358264 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:49.510105 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9326850-bc68-47a2-9116-4cf8ae5669cd None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:49.511765 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-c9326850-bc68-47a2-9116-4cf8ae5669cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:49.586057 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9326850-bc68-47a2-9116-4cf8ae5669cd None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:49.591526 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf19a32-09b0-495d-b319-01e5f61a166a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928d014648cd41d7b58e7e01845eb748', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:49.592189 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf19a32-09b0-495d-b319-01e5f61a166a service nova] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:49.592461 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf19a32-09b0-495d-b319-01e5f61a166a service nova] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:49.592734 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf19a32-09b0-495d-b319-01e5f61a166a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:49.594493 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-bcf19a32-09b0-495d-b319-01e5f61a166a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:49.606228 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf19a32-09b0-495d-b319-01e5f61a166a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:49.607870 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf19a32-09b0-495d-b319-01e5f61a166a service nova] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:49.618511 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1452/2910] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:09:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:09:49.954367 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=5 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:55.449340 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797e9cb-8f96-4264-9c34-45520e3f59b8 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:55.449970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797e9cb-8f96-4264-9c34-45520e3f59b8 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:55.449970 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797e9cb-8f96-4264-9c34-45520e3f59b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:55.508422 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-a797e9cb-8f96-4264-9c34-45520e3f59b8 None None] MFA Rules not processed for user `4d04998b636649d98103b1cd5f3a0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:55.509648 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a797e9cb-8f96-4264-9c34-45520e3f59b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:55.577476 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1459/2911] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jul 21 06:09:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:55.625762 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf19a32-09b0-495d-b319-01e5f61a166a service nova] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:55.627354 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-bcf19a32-09b0-495d-b319-01e5f61a166a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:55.722685 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf19a32-09b0-495d-b319-01e5f61a166a service nova] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:55.728274 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9754097-e161-4458-b56f-1044b861f593 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61b61188b5b94a51ba1ae1a88b78b89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:55.728924 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9754097-e161-4458-b56f-1044b861f593 service swift] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:55.729206 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9754097-e161-4458-b56f-1044b861f593 service swift] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:55.729483 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9754097-e161-4458-b56f-1044b861f593 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:55.738075 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9754097-e161-4458-b56f-1044b861f593 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:55.739710 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9754097-e161-4458-b56f-1044b861f593 service swift] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:55.748229 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1453/2912] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jul 21 06:09:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:09:56.266507 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fedccd-dba4-48aa-86e1-aab41179f552 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:56.266809 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fedccd-dba4-48aa-86e1-aab41179f552 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:56.267159 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fedccd-dba4-48aa-86e1-aab41179f552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:56.286495 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-01fedccd-dba4-48aa-86e1-aab41179f552 None None] MFA Rules not processed for user `4d04998b636649d98103b1cd5f3a0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:56.287831 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-01fedccd-dba4-48aa-86e1-aab41179f552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:56.296106 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1460/2913] 10.222.0.44 () {58 vars in 967 bytes} [Thu Jul 21 06:09:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:56.310340 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9754097-e161-4458-b56f-1044b861f593 service swift] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:56.321533 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9754097-e161-4458-b56f-1044b861f593 service swift] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:56.328040 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfd5ee-48d3-414f-a359-a814c9a7c3da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61b61188b5b94a51ba1ae1a88b78b89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:56.328695 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfd5ee-48d3-414f-a359-a814c9a7c3da service swift] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:56.328971 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfd5ee-48d3-414f-a359-a814c9a7c3da service swift] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:56.329273 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfd5ee-48d3-414f-a359-a814c9a7c3da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:56.337880 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfd5ee-48d3-414f-a359-a814c9a7c3da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:56.339517 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfd5ee-48d3-414f-a359-a814c9a7c3da service swift] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:56.347747 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1454/2914] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jul 21 06:09:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 21 06:09:56.554677 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b48fb-121b-49dd-bdd8-24126fa503ce None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:56.555135 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b48fb-121b-49dd-bdd8-24126fa503ce None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:56.555258 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b48fb-121b-49dd-bdd8-24126fa503ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:56.573555 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-f45b48fb-121b-49dd-bdd8-24126fa503ce None None] MFA Rules not processed for user `4d04998b636649d98103b1cd5f3a0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:56.574753 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f45b48fb-121b-49dd-bdd8-24126fa503ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:56.582163 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1461/2915] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jul 21 06:09:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:56.594107 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfd5ee-48d3-414f-a359-a814c9a7c3da service swift] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:56.605058 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfd5ee-48d3-414f-a359-a814c9a7c3da service swift] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:56.610359 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ef7b6-d261-4e5e-8258-d4e3acc96366 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61b61188b5b94a51ba1ae1a88b78b89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:56.610997 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ef7b6-d261-4e5e-8258-d4e3acc96366 service swift] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:56.611318 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ef7b6-d261-4e5e-8258-d4e3acc96366 service swift] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:56.611592 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ef7b6-d261-4e5e-8258-d4e3acc96366 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:56.619933 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ef7b6-d261-4e5e-8258-d4e3acc96366 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:56.621525 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ef7b6-d261-4e5e-8258-d4e3acc96366 service swift] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:56.629648 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1455/2916] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jul 21 06:09:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:09:56.960484 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c6cb6-888a-4c34-bd63-e826472bf394 None None] REQUEST_METHOD: `POST` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:56.960779 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c6cb6-888a-4c34-bd63-e826472bf394 None None] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:56.961055 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82c6cb6-888a-4c34-bd63-e826472bf394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:56.979791 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.auth.core [None req-f82c6cb6-888a-4c34-bd63-e826472bf394 None None] MFA Rules not processed for user `4d04998b636649d98103b1cd5f3a0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:56.981005 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-f82c6cb6-888a-4c34-bd63-e826472bf394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:56.989151 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1462/2917] 10.222.0.44 () {58 vars in 967 bytes} [Thu Jul 21 06:09:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:57.001368 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ef7b6-d261-4e5e-8258-d4e3acc96366 service swift] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:57.012467 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ef7b6-d261-4e5e-8258-d4e3acc96366 service swift] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:57.019351 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d09ef-8082-45ab-9223-4a359eba0cbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61b61188b5b94a51ba1ae1a88b78b89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:57.020001 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d09ef-8082-45ab-9223-4a359eba0cbf service swift] REQUEST_METHOD: `GET` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:57.020281 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d09ef-8082-45ab-9223-4a359eba0cbf service swift] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:57.020558 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d09ef-8082-45ab-9223-4a359eba0cbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:57.029372 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d09ef-8082-45ab-9223-4a359eba0cbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:57.031384 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d09ef-8082-45ab-9223-4a359eba0cbf service swift] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:57.039396 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1456/2918] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jul 21 06:09:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 21 06:09:59.119359 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c6cb6-888a-4c34-bd63-e826472bf394 None None] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:59.184020 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82c6cb6-888a-4c34-bd63-e826472bf394 None None] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:59.188950 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc3b7f-a4d0-459f-b9e5-ebd54476e7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:59.189812 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc3b7f-a4d0-459f-b9e5-ebd54476e7fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:59.190086 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc3b7f-a4d0-459f-b9e5-ebd54476e7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:59.190362 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc3b7f-a4d0-459f-b9e5-ebd54476e7fb admin admin] PATH_INFO: `/v3/users/3d7bbfba930843cd9782c4bf4931c3b1` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:59.191594 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc3b7f-a4d0-459f-b9e5-ebd54476e7fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d7bbfba930843cd9782c4bf4931c3b1)` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:59.193612 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc3b7f-a4d0-459f-b9e5-ebd54476e7fb admin admin] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:59.244412 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:59.246527 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8fbc3b7f-a4d0-459f-b9e5-ebd54476e7fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d7bbfba930843cd9782c4bf4931c3b1'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:59.262695 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8fbc3b7f-a4d0-459f-b9e5-ebd54476e7fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d7bbfba930843cd9782c4bf4931c3b1'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:59.269995 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8fbc3b7f-a4d0-459f-b9e5-ebd54476e7fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d7bbfba930843cd9782c4bf4931c3b1'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:59.270725 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8fbc3b7f-a4d0-459f-b9e5-ebd54476e7fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d7bbfba930843cd9782c4bf4931c3b1'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:59.271339 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.notifications [None req-8fbc3b7f-a4d0-459f-b9e5-ebd54476e7fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d7bbfba930843cd9782c4bf4931c3b1'} {{(pid=80390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:59.289754 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1463/2919] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jul 21 06:09:59 2022] DELETE /identity/v3/users/3d7bbfba930843cd9782c4bf4931c3b1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:09:59.294688 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84da8b7-a13d-43cf-8652-deb27d2f4f8c None None] REQUEST_METHOD: `POST` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:59.295233 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84da8b7-a13d-43cf-8652-deb27d2f4f8c None None] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:59.295485 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84da8b7-a13d-43cf-8652-deb27d2f4f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:59.315136 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.auth.core [None req-c84da8b7-a13d-43cf-8652-deb27d2f4f8c None None] MFA Rules not processed for user `3b3cc4a53587419894e0f708d1e60e89`. Rule list: `[]` (Enabled: `True`). {{(pid=80391) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 21 06:09:59.316348 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-c84da8b7-a13d-43cf-8652-deb27d2f4f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:59.325661 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=100 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:09:59.377012 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1457/2920] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jul 21 06:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 21 06:09:59.386991 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc3b7f-a4d0-459f-b9e5-ebd54476e7fb admin admin] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:59.388453 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-8fbc3b7f-a4d0-459f-b9e5-ebd54476e7fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:59.446873 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc3b7f-a4d0-459f-b9e5-ebd54476e7fb admin admin] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:59.450965 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a24cb-92fe-4b7f-839f-3525378ebf79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07508a9fa9146528620b7f8abec8549', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:59.451628 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a24cb-92fe-4b7f-839f-3525378ebf79 service neutron] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:59.451907 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a24cb-92fe-4b7f-839f-3525378ebf79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:59.452180 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a24cb-92fe-4b7f-839f-3525378ebf79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:59.461855 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a24cb-92fe-4b7f-839f-3525378ebf79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:59.463334 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a24cb-92fe-4b7f-839f-3525378ebf79 service neutron] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:59.473062 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1464/2921] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:09:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:09:59.644405 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84da8b7-a13d-43cf-8652-deb27d2f4f8c None None] Authenticating user token {{(pid=80391) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:09:59.646095 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.fernet_utils [None req-c84da8b7-a13d-43cf-8652-deb27d2f4f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80391) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:09:59.661593 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84da8b7-a13d-43cf-8652-deb27d2f4f8c None None] Validating token access rules against request {{(pid=80391) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:09:59.665083 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6f467-6053-4e56-ba55-d3e86bfde6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b3cc4a53587419894e0f708d1e60e89', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f7ec186b03924ec6a68eab5b0ea7d82d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80391) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:09:59.665966 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6f467-6053-4e56-ba55-d3e86bfde6e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:09:59.666246 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6f467-6053-4e56-ba55-d3e86bfde6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:09:59.666491 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6f467-6053-4e56-ba55-d3e86bfde6e1 admin admin] PATH_INFO: `/v3/projects/412742fe621f4e63acb5891ae57a10d0` {{(pid=80391) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:09:59.667764 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6f467-6053-4e56-ba55-d3e86bfde6e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=412742fe621f4e63acb5891ae57a10d0)` {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:09:59.670698 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6f467-6053-4e56-ba55-d3e86bfde6e1 admin admin] RBAC: Authorization granted {{(pid=80391) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:09:59.716972 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-25f6f467-6053-4e56-ba55-d3e86bfde6e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '412742fe621f4e63acb5891ae57a10d0', 'request_id': 'req-25f6f467-6053-4e56-ba55-d3e86bfde6e1'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:59.722862 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-25f6f467-6053-4e56-ba55-d3e86bfde6e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '412742fe621f4e63acb5891ae57a10d0', 'request_id': 'req-25f6f467-6053-4e56-ba55-d3e86bfde6e1'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:59.728197 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-25f6f467-6053-4e56-ba55-d3e86bfde6e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '412742fe621f4e63acb5891ae57a10d0', 'request_id': 'req-25f6f467-6053-4e56-ba55-d3e86bfde6e1'} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:59.742519 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-25f6f467-6053-4e56-ba55-d3e86bfde6e1 admin admin] The token cache is being invalidate because project 412742fe621f4e63acb5891ae57a10d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80391) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 21 06:09:59.742802 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-25f6f467-6053-4e56-ba55-d3e86bfde6e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:59.743433 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG keystone.notifications [None req-25f6f467-6053-4e56-ba55-d3e86bfde6e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80391) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 21 06:09:59.745125 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: [pid: 80391|app: 0|req: 1458/2922] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jul 21 06:09:59 2022] DELETE /identity/v3/projects/412742fe621f4e63acb5891ae57a10d0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 21 06:10:09.469005 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=39,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:10:09.739434 dv-cinder-850449-1-iscsi devstack@keystone.service[80391]: DEBUG dbcounter [-] [80391] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 21 06:12:18.182194 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a24cb-92fe-4b7f-839f-3525378ebf79 service neutron] Authenticating user token {{(pid=80390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 21 06:12:18.183795 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-a74a24cb-92fe-4b7f-839f-3525378ebf79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:12:18.328565 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a24cb-92fe-4b7f-839f-3525378ebf79 service neutron] Validating token access rules against request {{(pid=80390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 21 06:12:18.387652 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd2e78-710d-4d1f-92f1-3e130c8c9d41 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03d868598e39448881b43d299253d821', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b6f27b8d9b44d191210a24a9e04041', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 21 06:12:18.388305 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd2e78-710d-4d1f-92f1-3e130c8c9d41 service placement] REQUEST_METHOD: `GET` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 21 06:12:18.388572 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd2e78-710d-4d1f-92f1-3e130c8c9d41 service placement] SCRIPT_NAME: `/identity` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 21 06:12:18.388864 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd2e78-710d-4d1f-92f1-3e130c8c9d41 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 21 06:12:18.390932 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.fernet_utils [None req-89dd2e78-710d-4d1f-92f1-3e130c8c9d41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 21 06:12:18.403053 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd2e78-710d-4d1f-92f1-3e130c8c9d41 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 21 06:12:18.405024 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd2e78-710d-4d1f-92f1-3e130c8c9d41 service placement] RBAC: Authorization granted {{(pid=80390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 21 06:12:18.415349 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: [pid: 80390|app: 0|req: 1465/2923] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jul 21 06:12:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 21 06:12:28.410822 dv-cinder-850449-1-iscsi devstack@keystone.service[80390]: DEBUG dbcounter [-] [80390] Writing DB stats keystone:SELECT=89 {{(pid=80390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}